Why raw dense and BM25 scores should not be added directly

Dense retrieval ranks documents by the relationship between embedding vectors. Sparse BM25 retrieval ranks documents using term occurrences and document statistics. Even when both methods return numeric scores, those numbers represent different calculations and can vary with query wording, corpus composition, and implementation details.

Adding the two scores can make one retrieval method dominate for reasons unrelated to relevance. A tuning value that appears reasonable for one query set may behave differently after indexing new content or when users switch from concise keyword searches to longer natural-language questions. Before blending scores, establish whether their scales are genuinely comparable rather than merely numeric.

  • Dense retrieval can surface semantically related wording.
  • BM25 can reward exact terms, identifiers, and rare keywords.
  • Raw score ranges may change independently between the two systems.
  • A combined ranking should preserve useful signals from both methods.

Fuse ranked lists with Reciprocal Rank Fusion

RRF works from positions in result lists instead of from retrieval scores. Run the same query against dense search and BM25, retain a candidate list from each, and assign every document a fusion score based on its rank in each list. Documents found by both methods receive contributions from both rankings.

A common form is RRF(d) = Σ 1 / (k + rank_i(d)), where rank_i(d) is document d’s one-based position in retrieval list i and k is a positive constant. The constant reduces the gap between adjacent top ranks, so the fusion emphasizes broadly strong candidates rather than overreacting to a small rank difference.

  • Use a stable document identifier to deduplicate candidates across lists.
  • Treat a missing document in a list as contributing zero from that list.
  • Choose candidate depths that leave enough results for fusion and later filtering.
  • Apply deterministic tie-breaking, such as a document ID, for repeatable responses.

Implement the pattern at the API boundary and evaluate it

For an API-first application using Talqora’s dense and sparse retrieval components, a service layer can issue the dense and BM25 requests, collect their ranked identifiers, and calculate RRF before returning a unified response. Keeping fusion in this layer makes the policy explicit: clients receive one result shape while the application retains control over candidate depth, rank constants, and metadata filters.

Evaluation should use representative queries rather than relying on score distributions alone. Include exact-name lookups, technical terminology, broad conceptual questions, and queries with ambiguous language. Review whether relevant documents appear in the candidate lists and whether fusion improves their final positions. If not, inspect the source rankings first: fusion cannot recover a document that neither retriever returned.

  • Log per-source rank alongside the fused rank for debugging.
  • Test metadata and access-control filtering before exposing final results.
  • Compare dense-only, BM25-only, and fused rankings on the same judged query set.
  • Revisit candidate depth when content, query patterns, or filtering rules change.