Why dense and sparse scores should not be added directly

Talqora uses regional S3 Vectors for dense search and Quickwit BM25 for sparse search. These retrieval methods produce rankings from different representations: dense retrieval compares embeddings, while BM25 ranks documents from term statistics and query-term matches.

A tempting implementation is to normalize both score sets and add them together. That approach is fragile because score distributions can change with corpus composition, query length, embedding model choice, analyzer configuration, and each engine's scoring behavior. A score that looks high in one ranking is not necessarily equivalent to the same-looking score in the other.

Rank-based fusion avoids requiring semantic equivalence between scores. It asks a simpler question: how highly did each retrieval method rank a candidate?

  • Dense retrieval helps recover semantic paraphrases and related phrasing.
  • BM25 helps preserve exact-token matches, including names, codes, and rare terms.
  • Rank positions are easier to combine than raw scores from separate retrieval systems.

Fuse candidate lists with reciprocal rank fusion

For a query, retrieve a bounded candidate list from the dense path and another from the BM25 path. Deduplicate documents by a stable document identifier, then assign every candidate an RRF score. For each ranking where a document appears, add 1 divided by k plus its one-based rank.

In formula form, RRF(d) = Σ 1 / (k + rank_i(d)). The sum runs over the rankings that contain document d. The constant k reduces the difference between adjacent top ranks, preventing one list from dominating simply because it has a steep internal score curve.

For example, if a document ranks 2nd in dense retrieval and 8th in BM25, it receives contributions from both lists. A document found only by one method can still rank well, but agreement across methods is rewarded naturally.

  • Use one-based rank positions: 1 is the top result.
  • Choose the same candidate depth for both paths as a simple starting point.
  • Use a stable ID for deduplication before sorting fused candidates.
  • Keep the original ranks for debugging and result explanations.

Make fusion observable before tuning it

Start with a small evaluation set built from real search intents. Include queries with exact identifiers, short ambiguous phrases, long natural-language questions, and terminology that has close synonyms. For each query, inspect dense-only, BM25-only, and fused results side by side.

The most useful diagnostic is provenance: record whether a final result came from dense retrieval, BM25 retrieval, or both, along with its rank in each source list. This reveals whether fusion is broadening recall, whether one path contributes little, and whether relevant documents are being lost before fusion because the candidate depth is too shallow.

Tune one variable at a time. Candidate depth and the RRF constant affect different things: depth controls which documents are eligible for fusion, while k controls how quickly rank contributions decay. Change either only after preserving a query set and reviewing the resulting ranking differences.

  • Log query text, document ID, source ranks, and fused rank.
  • Separate evaluation slices for exact-match and semantic queries.
  • Investigate relevant documents absent from both candidate lists before changing fusion logic.
  • Treat RRF as a transparent baseline, not as a substitute for relevance evaluation.