Why raw-score merging is fragile

Dense search returns results based on the relationship between a query embedding and stored embeddings. BM25 returns results based on term occurrence, term rarity, and document-length normalization. Both produce rankings, but their numeric scores have different meanings.

Adding, averaging, or thresholding those raw scores can create unstable behavior. A change to an embedding model, vector similarity configuration, analyzer, or corpus distribution may shift score ranges without changing the underlying relevance quality. Ranking positions are often a safer common signal.

  • Dense retrieval is useful for semantic similarity and paraphrased language.
  • BM25 is useful for exact terms, identifiers, product names, and rare vocabulary.
  • A score of 0.8 in one retrieval system does not inherently equal 0.8 in another.
  • Fusion should preserve evidence from both result lists.

Fuse ranked lists with reciprocal rank fusion

Reciprocal rank fusion, or RRF, combines result lists by assigning each document a contribution based on its rank. For every document returned by either retrieval path, add 1 divided by k plus its one-based rank. The document's final score is the sum of its contributions across all lists.

In practice, retrieve a bounded candidate set from S3 Vectors and another from Quickwit BM25, join them by a stable document or chunk identifier, then sort by the RRF total. The constant k reduces the impact of small differences near the first rank while still rewarding documents that rank well in either or both systems.

  • Formula: RRF(document) = Σ 1 / (k + rank).
  • Use one-based ranks: the top result has rank 1.
  • Choose and document a fixed k before evaluating changes.
  • Treat a missing document from one list as contributing zero from that list.

Make fusion operationally reliable

RRF depends on matching the same logical content across dense and sparse indexes. Store a stable canonical identifier for each retrievable chunk, rather than relying on independently generated IDs. If content is re-chunked or replaced, update both retrieval paths together so that fusion does not combine stale and current versions.

Evaluate the fused ranking using representative queries that include exact names, abbreviations, multi-word concepts, and natural-language questions. Inspect not only whether the right document appears, but also whether it appears early enough for the calling application. Keep dense-only and BM25-only rankings in logs or evaluation output so regressions remain diagnosable.

  • Use identical canonical chunk IDs in dense and sparse records.
  • Apply the same tenant, access, language, and freshness filters before fusion.
  • Deduplicate results after joining candidate lists.
  • Measure dense-only, sparse-only, and fused rankings separately.