Weighted RRF and the vector_top_k trap

Reciprocal Rank Fusion is the simplest way to merge results from searches that don't compare, like cosine and BM25: each leg says in which position it placed each item, and the score is the sum of 1 / (k + position). With k = 60 it works almost every time. Here are the two cases where it didn't for me, in a RAG engine on Turso with three legs: vector, terms (FTS5 with OR) and exact phrase (FTS5 with the phrase in quotes).

RRF flattens the strongest signal

RRF only looks at position, which is why it works for cosine and BM25, which live on different scales. But do the maths with k = 60:

• 1st place is worth 1/61 and 2nd is worth 1/62. The gap is 1.6%.
• Being 40th in two legs (1/100 + 1/100 = 2/100) beats being 1st in a single leg (1/61).

For vectors and loose terms, that's fine. The problem is the phrase leg: a document containing exactly the phrase the person typed, in order, is the strongest signal there is, and plain RRF treats it like the others.

The fix was a weight:

const RRF_K = 60;
const PHRASE_WEIGHT = 2;

const raw =
  (vectorRank === null ? 0 : 1 / (RRF_K + vectorRank)) +
  (textRank === null ? 0 : 1 / (RRF_K + textRank)) +
  (phraseRank === null ? 0 : PHRASE_WEIGHT / (RRF_K + phraseRank));

Why 2? It's the smallest weight that works. With it, the phrase in 1st place is worth 2/61, and the best possible combination of the other two legs is 1/61 + 1/61, a tie. In practice the other legs never both put the same item first, so the phrase wins. And a poorly placed phrase (30th, say) can't override the other two legs' consensus on its own. A bigger weight would.

Normalising by the legs that answered

Not every question activates all three legs. The phrase only exists with two words or more, and the term leg can come back empty. To keep the final score between 0 and 1, I divide by the maximum possible with the legs that actually answered:

const activeLegs =
  (vectorHits.length > 0 ? 1 : 0) +
  (textHits.length > 0 ? 1 : 0) +
  (phraseHits.length > 0 ? PHRASE_WEIGHT : 0);
const maxRrf = Math.max(activeLegs, 1) / (RRF_K + 1);
const score = raw / maxRrf;

Without it, the same relevance would get different scores depending on how many legs ran, and any cut-off ("only show above 0.4") would be wrong.

The vector_top_k trap

With more documents, I replaced the full scan with libSQL's vector index:

CREATE INDEX chunk_emb_idx ON chunk (libsql_vector_idx(emb));

SELECT c.id, 1 - vector_distance_cos(c.emb, vector32(?)) AS similarity
  FROM vector_top_k('chunk_emb_idx', vector32(?), ?) AS v
  JOIN chunk c ON c.rowid = v.id
  JOIN document d ON d.doc_id = c.doc_id
 WHERE d.doc_id IN (?, ?, ?)
 ORDER BY similarity DESC;

It looks right and returns nothing. vector_top_k finds the k nearest neighbours across the whole index, and only then does WHERE filter. If the search is scoped to a few documents and k is small, the top k of the whole database belong to other documents, the filter removes all of them, and the answer is zero rows, with no error.

The fix is to widen the window when there's a filter:

const MAX_CANDIDATES = 200;

export function candidateWidth(limit: number, scoped: boolean): number {
  if (scoped) return MAX_CANDIDATES;
  return Math.min(Math.max(limit * 8, 1), MAX_CANDIDATES);
}

Without a filter, eight times what I need, capped. With a filter, straight to the cap. It isn't perfect: a very small scope in a very large database can still come back empty, and then the right move is a full scan of just that scope. But it solved the case I had, and the probe I use to check it runs exactly the scenario that failed: a scoped search with a limit of 6.

If you use RRF, do the maths with real positions before trusting it. And in your vector index's docs, check whether the filter runs before or after the search.