Why raw dense and sparse scores should not be mixed

A dense-search score and a BM25 score are produced by different retrieval models and have different ranges, distributions, and meanings. A high BM25 score may reflect repeated or rare term matches; a high dense-search score reflects the similarity measure used by the embedding workflow. Adding the two values together without calibration can make ranking sensitive to index settings, query length, or changes in embedding models.

This becomes visible on mixed queries. A user may ask for an error code plus a natural-language description. BM25 may rank the exact code highly, while dense retrieval may find documentation that explains the underlying issue using different wording. Both lists carry useful evidence, but their numeric scores should not be assumed to be interchangeable.

  • Use raw score addition only when scores have been intentionally normalized and validated together.
  • Keep the dense and sparse candidate lists separate through the first retrieval step.
  • Evaluate hybrid behavior on queries containing both exact terms and semantic intent.

Fuse ranked lists with Reciprocal Rank Fusion

RRF ignores the magnitude of each retrieval score and instead rewards documents that appear near the top of one or more ranked lists. For each document d, calculate RRF(d) = Σ 1 / (k + rank_i(d)), where rank_i(d) is the document position in retrieval list i and k is a positive constant chosen by the application.

In a Talqora-oriented retrieval path, issue a dense query against the S3 Vectors-backed index and a sparse query against Quickwit BM25. Collect a candidate window from each result set, deduplicate by a stable document or chunk identifier, calculate the fused rank, and return the highest-scoring documents. A document found by both retrieval methods gains evidence from both lists without requiring their underlying scores to match.

  • Choose a stable identifier so the same chunk from both result sets is merged correctly.
  • Retrieve more candidates than the final response needs before fusion.
  • Use the same k value during evaluation so result changes are attributable to retrieval changes.
  • Retain each source rank in logs for debugging and relevance review.

Treat fusion as a measurable retrieval stage

RRF is deliberately simple, but it still has operational choices. Candidate depth determines whether a document can receive support from both retrievers. Chunking affects whether exact tokens and semantic context remain in the same retrievable unit. The fusion constant affects how quickly the advantage of a top-ranked item declines as rank increases.

Build a small query set from real search intent: known-item lookups, terminology-heavy questions, paraphrased questions, and queries containing product names or identifiers. Review whether the fused results preserve exact-match documents while adding useful semantic neighbors. When relevance failures occur, inspect the separate dense and BM25 ranks before changing the fusion formula; the issue may instead be chunk boundaries, metadata filtering, or query construction.

  • Log dense rank, BM25 rank, fused rank, and the final selected chunk ID.
  • Test candidate depths and chunking choices before introducing more complex ranking logic.
  • Apply required metadata or tenant filters consistently to both retrieval paths.
  • Use judged queries to decide whether hybrid retrieval improves the result set for your use case.