Why raw dense and sparse scores should not be added blindly

A dense search result is typically ordered by a vector-similarity measure, while BM25 is a lexical relevance function based on term statistics. Even when both outputs are numeric, the meaning and range of those numbers depend on the retrieval method, index configuration, query shape, and corpus.

Adding those values with a fixed weight can work in a narrowly tested situation, but it makes ranking sensitive to score-distribution changes. A longer query, a rare token, or a different embedding model can alter one score scale without changing the underlying relevance judgment. Rank-based fusion avoids requiring a shared score interpretation.

  • Dense retrieval helps when relevant documents use different vocabulary.
  • BM25 helps when exact words, identifiers, and rare terms matter.
  • A numeric score is not automatically a calibrated relevance probability.
  • Keep each retriever responsible for producing its own ordered candidate list.

Fuse two candidate lists with reciprocal rank fusion

In Talqora's retrieval design, dense candidates can come from regional S3 Vectors and sparse candidates from Quickwit BM25. Request a bounded top-K list from each path, deduplicate by document ID, and assign every document an RRF score based on its position in each list.

For a document d, the common form is RRF(d) = Σ 1 / (k + rank_i(d)). The sum runs across retrieval lists that contain d. The constant k softens the advantage of the first few positions; choose it deliberately and keep it stable while evaluating changes. A document found near the top by both retrievers receives a stronger combined signal than one found by only one.

  • Retrieve dense top-K and BM25 top-K independently.
  • Use document IDs to merge and deduplicate candidates.
  • Compute one contribution per list in which a document appears.
  • Sort by fused score, then apply deterministic tie-breaking such as document ID.

Make fusion operationally safe and measurable

Start with a small, representative query set that includes semantic questions, exact-phrase queries, identifier lookups, and mixed queries. Record the candidate lists before fusion as well as the final ordering. This makes it possible to tell whether a disappointing result came from dense recall, BM25 recall, or the fusion step itself.

Treat regional placement as part of the retrieval path: send a query to the intended regional resources and keep the fusion service aware of which result lists belong to the same corpus and filtering rules. If the two retrievers search different document populations or apply different access constraints, fusion can produce confusing results regardless of the formula.

  • Apply the same tenant, access, deletion, and document-type constraints to both paths.
  • Log ranks and source membership, not only the final fused score.
  • Evaluate candidate depth separately from final ranking quality.
  • Add a reranking stage only after the fused candidate set is understood.