Why raw dense and BM25 scores should not be added blindly

Dense retrieval scores reflect the relationship between an embedded query and embedded documents. BM25 scores reflect token-level term matching and document statistics. Even when both are useful relevance signals, their numeric ranges and behavior are not inherently aligned.

A simple weighted sum can therefore be fragile. A change in embedding model, query length, corpus composition, or sparse-index settings may shift one score distribution enough to dominate the other, even if the intended relevance balance has not changed.

  • Dense retrieval can surface paraphrases and conceptually related text.
  • BM25 can reward exact identifiers, rare terms, and literal phrases.
  • Score magnitude alone does not establish comparable relevance across retrieval methods.

Retrieve independently, then fuse ranks

Start by issuing the dense and sparse retrieval requests as separate candidate-generation steps. Keep a modest top-k list from each path, deduplicate documents by a stable document identifier, and combine the lists only after each retriever has produced its own ranking.

Reciprocal rank fusion (RRF) is a practical baseline because it uses rank positions rather than raw scores. For each document, add a contribution based on its position in each list. Documents returned by both retrievers naturally receive more support, while a strong result from only one retriever can still remain competitive.

  • Choose a candidate depth that gives both retrievers room to contribute.
  • Use stable IDs when deduplicating results from dense and sparse paths.
  • Record source ranks so fused results remain explainable during debugging.
  • Apply deterministic tie-breaking, such as a document ID, when ranks are equal.

Validate fusion with representative queries

Evaluate the fused ranking against query groups that reflect real retrieval needs. Include exact-name or part-number queries, natural-language questions, acronym-heavy requests, and short ambiguous queries. These groups reveal whether one retriever is being unintentionally suppressed.

Inspect failures before changing fusion parameters. If exact strings are missing from the sparse candidate list, fusion cannot recover them. If semantically relevant documents never reach the dense candidate list, changing the final ranking formula will not solve the underlying recall issue.

  • Compare dense-only, BM25-only, and fused result lists for the same queries.
  • Track whether relevant documents enter either candidate list before judging final ranking.
  • Review query classes separately instead of relying on one aggregate relevance measure.
  • Revisit candidate depth and fusion settings whenever retrieval models or indexes change.