Why raw dense and BM25 scores should not be added directly

Dense retrieval returns documents based on proximity between embeddings. BM25 ranks documents using term occurrence, term rarity, and document-length normalization. Both outputs are useful, but their score values arise from different calculations and do not inherently represent the same scale.

Adding a dense similarity score to a BM25 score can make one retriever dominate for accidental numerical reasons. A small change in embedding model, distance metric, corpus composition, or BM25 configuration can alter score distributions without indicating that either retrieval method has become more or less relevant.

Rank-based fusion avoids this calibration problem. It uses the ordering produced by each retriever, which is the signal each system is explicitly designed to provide.

  • Dense retrieval can surface semantic matches with little lexical overlap.
  • BM25 can strongly reward exact names, codes, phrases, and rare terms.
  • Score scales may change across indexes or retrieval configurations.
  • A shared document identifier is required to merge result sets.

Apply reciprocal rank fusion to two result lists

For a query, retrieve a candidate list from regional S3 Vectors and another from Quickwit BM25. Assign each document a fusion score using RRF: score(d) = Σ 1 / (k + rank_i(d)), where rank_i(d) is the one-based position of document d in retrieval list i. Documents absent from a list contribute nothing from that list.

The constant k reduces the difference between adjacent top ranks and prevents rank one from overwhelming every lower-ranked result. A commonly used starting value is 60, but it is a tuning parameter rather than a universal rule. Evaluate it against representative queries and relevance judgments when those are available.

After summing contributions from dense and sparse lists, sort documents by the fused score. A document that ranks well in both lists rises naturally, while a strong result unique to one retrieval method can still remain competitive.

  • Request more candidates than the final number of results you plan to display.
  • Deduplicate by stable document or chunk ID before producing the final ranking.
  • Use one-based ranks consistently in the RRF formula.
  • Keep the source ranks during debugging to explain why a result appeared.

Make fusion reliable with consistent document identity

Fusion quality depends on recognizing that a dense result and a BM25 result refer to the same retrievable unit. Index the same stable ID in both systems, ideally at the chunk level if search returns chunks rather than whole documents. Store metadata needed for filtering and presentation alongside that identity in the relevant retrieval workflow.

Choose chunking deliberately. If sparse indexing uses one document per page while dense indexing uses several chunks per page, RRF will combine unlike units. That may be appropriate in some applications, but it should be a conscious design choice, followed by grouping or deduplication before the user sees results.

Operationally, log the query, dense rank, BM25 rank, fused rank, and selected ID. These records make it easier to identify queries where exact terminology should matter more, where semantic recall is weak, or where duplicate chunks occupy too many positions.

  • Use deterministic IDs across dense and sparse indexing pipelines.
  • Version chunking and embedding changes so result shifts are traceable.
  • Apply the same access-control and tenant filters to both candidate sets.
  • Inspect fused results with query classes such as names, error codes, concepts, and natural-language questions.