Why dense and sparse scores should not be added blindly

Dense retrieval ranks documents by proximity between an embedded query and embedded content. It is useful when a query and a relevant document use different wording but express a similar idea. Talqora’s dense path uses regional S3 Vectors.

BM25 ranks documents from lexical evidence: the query terms found in a document, their frequency, and their distribution across the collection. Quickwit BM25 can therefore surface documents that contain an important identifier, product name, error code, or exact phrase.

These scores are not automatically comparable. Their ranges, distributions, and meaning depend on the retrieval method and query. Adding them together can cause one retriever to dominate for reasons unrelated to relevance.

  • Use dense retrieval for semantic similarity and paraphrases.
  • Use BM25 for exact terminology, rare tokens, and literal matches.
  • Treat each system’s returned ordering as the stable shared signal.

Fuse ranked lists with reciprocal rank fusion

Reciprocal rank fusion, commonly called RRF, assigns each document a contribution based on its position in every result list. For a document d, calculate RRF(d) as the sum of 1 divided by k plus its rank in each list where it appears. The constant k reduces the impact of a single top position and makes the fusion less sensitive to small rank changes.

In an application retrieval layer, issue the same user query to the dense and BM25 paths, request a candidate set from each, and merge results by a stable document identifier. A document found by both paths receives two contributions; a document found by only one path can still remain in the final ranking.

RRF does not require dense and BM25 scores to be normalized. It uses only ranks, which makes it a useful default when the two search systems expose different scoring behavior.

  • Keep the original rank position from each result list.
  • Deduplicate candidates using a canonical document or chunk ID.
  • Compute the fused score after both searches return.
  • Sort by fused score, then apply a deterministic tie-breaker.

Design the candidate set and validate with real queries

Fusion can only rank documents that enter the candidate pool. Request enough candidates from both dense and sparse retrieval so that useful results beyond the very top positions can participate. The right depth depends on corpus size, document chunking, and the number of results your application ultimately presents.

Evaluate with a query set that reflects actual usage. Include semantic requests, exact-name lookups, acronym-heavy queries, partial error messages, and ambiguous questions. For each query, record whether a relevant document appears in the fused top results and whether either individual retriever supplied it.

This analysis is especially valuable for deciding where to invest next. If BM25 consistently rescues identifier-based searches, preserve sparse recall. If dense retrieval finds relevant paraphrases that lexical search misses, preserve the embedding path. Fusion makes those complementary contributions visible.

  • Test chunk-level and document-level identifiers deliberately.
  • Inspect queries where only one retrieval path produced the relevant result.
  • Use a fixed evaluation set before changing candidate depth or fusion settings.
  • Log retriever ranks alongside the final fused rank for debugging.