Why ranks are safer to combine than raw scores

Dense search and BM25 produce scores with different meanings. A dense similarity score depends on the embedding model and similarity method, while a BM25 score depends on term statistics and document length. Treating the two values as directly interchangeable can make a combined ranking sensitive to index settings and query composition.

RRF avoids that comparison. Instead of combining raw scores, it uses each result's position in its own ranked list. A document that ranks highly in either retrieval path receives a useful contribution, while a document found by both paths can accumulate support.

  • Use dense retrieval for semantic candidate discovery.
  • Use BM25 for exact terminology, codes, names, and uncommon terms.
  • Keep each retrieval path independently observable before combining results.

Retrieve two candidate lists, then fuse them

For a query, generate an embedding and request a ranked dense candidate list from the vector store. In parallel, send the text query to the BM25 index for a sparse candidate list. The lists do not need to contain the same documents, but they must use stable document identifiers so that duplicates can be recognized during merging.

For every document d, RRF assigns a fused score: RRF(d) = Σ 1 / (k + rank_i(d)). The sum runs across the ranked lists in which the document appears. The constant k reduces the difference between nearby ranks and prevents the first few positions from dominating too sharply.

  • Choose a candidate depth large enough to give fusion useful overlap and coverage.
  • Deduplicate by a canonical document or chunk identifier before returning results.
  • Record each source rank alongside the fused score for debugging.
  • Treat k and candidate depth as evaluation parameters, not universal constants.

Use regional dense and sparse indexes as complementary paths

In a Talqora deployment, the dense path can be backed by regional S3 Vectors, while the sparse path can use Quickwit BM25. The application layer can issue or coordinate the two retrieval requests, apply RRF to their ranked outputs, and return a single ordered candidate set to a reranker or answer-generation workflow.

Evaluate the fused result set with queries drawn from real user behavior. Include semantic paraphrases, exact product terms, identifiers, misspellings, and mixed queries. The goal is not to prove that one retrieval method always wins; it is to identify where the two methods cover each other's blind spots.

  • Log query text, retrieval source, rank, and selected document ID.
  • Inspect queries where only one path found the expected result.
  • Revisit chunking and metadata filters when relevant documents are retrieved but poorly ordered.
  • Keep a small labeled query set to test changes to embeddings, analyzers, and fusion settings.