Why raw dense and sparse scores should not be added

A dense retrieval score reflects the similarity function used by an embedding model and vector index. A BM25 score reflects term frequency, document frequency, field length, and query-term matches. Even when both scores are ordered correctly within their own result lists, their numeric ranges and distributions are not inherently comparable.

A fixed formula such as dense_score + bm25_score can therefore produce unstable behavior. One retriever may dominate simply because its scores occupy a larger numeric range, not because its candidates are more useful for the query. Normalizing scores can help in some systems, but it introduces another set of assumptions that must be monitored as corpora, embedding models, and query mixes change.

  • Dense search is useful for semantic paraphrases and concept-level matches.
  • BM25 is useful for exact vocabulary, codes, names, and rare terms.
  • Raw score magnitudes are retriever-specific rather than universal relevance values.
  • Rank positions are often a safer common signal for an initial hybrid design.

Fuse candidate lists with reciprocal rank fusion

RRF assigns each document a contribution based on its position in each ranked list. For a document d, the fused score is the sum of 1 divided by k plus its rank in every list where it appears. The constant k reduces the advantage of being first by a small margin and makes the method less sensitive to small rank changes near the top.

For Talqora-style retrieval, request a candidate list from dense search backed by regional S3 Vectors and another from Quickwit BM25. Deduplicate by a stable document or chunk identifier, calculate the RRF score, sort descending, and return the top fused results. A document found by both methods gains two contributions; a document found only by one method can still rank well if it appears near the top.

  • Use the same query text for both searches unless your application has a deliberate query-rewriting step.
  • Retrieve a wider candidate set than the final page size, such as the top N results from each retriever.
  • Start with a conventional k value such as 60, then evaluate it against representative queries.
  • Preserve source ranks and fused scores in logs so ranking decisions can be inspected.

Evaluate fusion on the queries users actually ask

RRF is simple, but it is still a ranking policy and should be evaluated. Build a small query set from production-like search intents: exact identifier lookups, terminology-heavy queries, natural-language questions, short ambiguous queries, and queries that depend on metadata filters. For each query, record whether the needed document or chunk appears in the top results.

Compare dense-only, BM25-only, and fused rankings. The goal is not to prove that hybrid retrieval wins every query. Instead, identify which query classes each approach serves and verify that fusion improves coverage without creating unacceptable regressions. When reviewing failures, inspect the underlying chunks as well as ranks: poor chunk boundaries, stale content, and missing metadata can look like ranking problems.

  • Measure recall-oriented signals such as whether a known relevant item appears in the top 5 or top 10.
  • Segment results by query type rather than relying on one aggregate number.
  • Test filtered retrieval separately because filters can change the available candidate pool.
  • Re-run the evaluation after changing embedding models, chunking rules, corpus content, or BM25 settings.