Why raw dense and sparse scores should not be mixed

A dense retrieval result is ordered by a vector similarity measure, while a BM25 result is ordered by a term-based relevance function. Even when both lists are relevant, a score of 0.8 from one system does not have a general, portable relationship to a score of 8 from another. Treating them as equivalent can make ranking behavior unstable as document collections, queries, or engine settings change.

Score normalization can help in tightly controlled experiments, but it introduces more assumptions. You must choose a normalization method, decide its scope, and monitor how score distributions shift over time. For many production retrieval paths, ranking positions are a simpler and more robust input to a first hybrid implementation.

  • Dense search helps with paraphrases, related concepts, and semantic intent.
  • BM25 helps with exact phrases, error codes, product names, and uncommon terms.
  • Raw scores should be considered engine-specific unless they have been explicitly calibrated.
  • Keep the source rank and source-specific score for debugging, even if fusion uses rank.

Apply reciprocal rank fusion to two result lists

RRF assigns each document a contribution based on its position in each result list. For a document at rank r, the contribution is 1 divided by k plus r. The final fused score is the sum of that contribution across the dense and sparse lists. The constant k reduces the influence of very high positions and makes the fusion less sensitive to small rank differences near the top.

For example, request a candidate list from regional S3 Vectors and another from Quickwit BM25. Match results using a stable document or chunk identifier. If a chunk appears in both lists, it receives two contributions; if it appears in only one, it can still be included in the merged result set. Sort by fused score and return the top results.

  • Use the same stable ID for the dense and sparse representations of a retrievable chunk.
  • Start with equal treatment of the two rank lists before adding query-specific weighting.
  • Choose a fixed k and document it so ranking behavior is reproducible.
  • Fetch more candidates than the final response size to give the fusion step useful overlap.

Make hybrid retrieval observable and testable

A fused ranking is easier to improve when each returned result carries retrieval provenance. Record whether a result came from dense retrieval, BM25 retrieval, or both, along with its ranks in each list and its final fused score. This makes it possible to inspect failures such as an exact identifier being buried by semantically similar but incorrect content.

Evaluate the fused path with a query set that reflects real traffic. Include natural-language questions, short keyword searches, identifiers, abbreviations, and queries with spelling or terminology variation. Review not only whether a relevant item appears, but also whether it appears high enough for the downstream application or user interface.

  • Log query text, candidate IDs, source ranks, and the final fusion order.
  • Build test cases for exact-match queries and paraphrased queries separately.
  • Check for duplicate chunks before presenting results to users.
  • Use retrieval traces to decide whether a failure came from indexing, candidate generation, or fusion.