Why fuse ranks instead of raw scores?

Dense-search similarity values and BM25 scores are not naturally comparable. Their ranges can vary with embedding models, index settings, document length, query composition, and the retrieval engine itself. Adding those raw values together can make a ranking sensitive to arbitrary scaling choices.

RRF avoids score calibration by using only each document's position in a result list. A document receives credit when it ranks highly in either retriever, and receives additional credit when both retrievers independently place it near the top.

  • Dense retrieval helps with paraphrases and conceptual similarity.
  • BM25 helps preserve exact-token signals such as error codes, SKUs, and names.
  • Rank-based fusion keeps the two scoring systems separate.
  • The method is useful when relevance labels for learned reranking are unavailable.

Implement RRF as an application-side merge

For each query, request a bounded candidate list from the dense path and another from the sparse path. Assign ranks starting at 1. For every document ID appearing in either list, accumulate the RRF contribution 1 / (k + rank) from each list where it appears. Sort documents by the accumulated value and return the top results.

The constant k reduces the difference between nearby ranks. A commonly used starting point is 60, but it is a tuning parameter rather than a universal rule. Keep the original dense and BM25 ranks with the fused result so that relevance investigations remain explainable.

  • Use the same canonical document or chunk ID in both indexes.
  • Retrieve enough candidates from each path to create meaningful overlap and coverage.
  • Deduplicate by canonical ID before presenting results.
  • Store per-source rank and fused score in retrieval logs.

Tune with a representative query set

Start by assembling queries that reflect real retrieval behavior: natural-language questions, exact identifiers, short keyword searches, and ambiguous requests. For each query, inspect whether the desired document is found by dense search, BM25, both, or neither. This diagnosis is more useful than treating hybrid retrieval as a single black box.

Then vary candidate depth and the RRF constant while checking ranked results against judged examples or careful human review. If neither candidate list contains the relevant item, fusion cannot recover it; improve chunking, metadata filters, indexing coverage, query construction, or the underlying retrievers before changing fusion logic.

  • Evaluate exact-term and semantic queries separately before combining results.
  • Watch for chunk-level duplicates that can crowd out distinct documents.
  • Apply authorization and tenant filters consistently to both retrieval paths.
  • Use a later reranking stage only after candidate recall is acceptable.