Why raw-score blending is fragile

Dense and sparse systems produce scores for different reasons. A dense-search score reflects the relationship between embedded query and document vectors. A BM25 score is based on term frequency, document frequency, and document-length effects. Even when both scores are higher for better results, their numeric ranges and distributions need not align.

A weighted formula such as dense_score + sparse_score can therefore behave unpredictably. A change to an embedding model, chunking strategy, BM25 configuration, or corpus composition may alter one score distribution without changing relevance in the same proportion. Maintaining a stable weight then becomes an ongoing calibration task.

  • Do not assume a dense similarity value and a BM25 score share a common meaning.
  • Avoid selecting fusion weights solely from a small set of hand-picked queries.
  • Treat retrieval changes such as re-embedding or re-chunking as possible score-distribution changes.
  • Keep dense and sparse result lists available for debugging, even after fusion.

Fuse ranks instead of scores

Reciprocal Rank Fusion combines result lists using position rather than raw score. For each document in each list, add 1 divided by k plus its rank. The constant k reduces the difference between nearby high ranks, preventing a single first-place result from overwhelming all other evidence.

In a Talqora-backed retrieval flow, an application can issue a dense query against regional S3 Vectors and a sparse query through Quickwit BM25, request a fixed candidate depth from each, then fuse the returned document identifiers. A document retrieved by both paths gains evidence from both lists; a document highly ranked by only one path can still remain competitive.

  • Use one-based ranks: rank 1 is the first result in a list.
  • Compute: RRF(document) = sum of 1 / (k + rank) across lists containing the document.
  • Start with the same candidate depth for dense and sparse retrieval to simplify inspection.
  • Deduplicate by a stable document or chunk identifier before returning results.

Make fusion observable and query-aware

RRF is simple, but it should not be invisible. Log the dense rank, sparse rank, fused rank, and retrieval path for each returned item. These fields make it possible to see whether a result was supported by both signals or promoted by only one. They also help diagnose failures caused by missing metadata filters, duplicate chunks, or a query that contains an important exact token.

Evaluate with a query set that reflects real traffic patterns: natural-language questions, short keyword searches, identifier-heavy queries, and ambiguous terms. Inspect not only whether a relevant item appears, but also whether it appears early enough for the next stage of the application. If some query classes consistently depend on one retrieval mode, that is useful evidence for routing or per-query weighting later.

  • Record source ranks and fused scores alongside query and filter context.
  • Review queries where dense-only and BM25-only top results disagree sharply.
  • Test candidate depth separately from final result count; fusion cannot rescue documents that were never retrieved.
  • Apply the same authorization and metadata-filtering rules to both retrieval paths before fusion.