Why raw dense and sparse scores should not be added

A dense search score and a BM25 score are produced by different retrieval models and indexing systems. Their numerical ranges, distributions, and sensitivity to query length can differ substantially. A score of 0.7 from one system is not inherently equivalent to a score of 0.7, 7, or 70 from another.

Adding uncalibrated scores can make a hybrid ranker fragile. A change in embedding model, corpus composition, analyzer settings, or sparse-query formulation may alter one score distribution and unexpectedly dominate the final order. Rank-based fusion avoids assuming that either system's raw score is globally comparable.

  • Dense retrieval is useful for semantic similarity and paraphrases.
  • BM25 is useful for exact vocabulary, codes, names, and rare terms.
  • Score ranges are implementation-specific rather than universal relevance units.
  • Use each retriever's ordering as the stable signal when score calibration is unavailable.

Apply reciprocal rank fusion to two result lists

Run dense search and BM25 search independently, then retain a ranked candidate list from each. For every document appearing in either list, calculate an RRF contribution from each rank list: 1 divided by k plus the document rank. Sum the contributions across lists, then sort documents by the combined value.

The constant k reduces the difference between adjacent top ranks and makes fusion less dominated by a single rank-one placement. It is a tunable policy parameter, not a relevance guarantee. Start with a documented default, evaluate against representative queries, and keep the same rule across experiments so changes are attributable.

  • Formula: RRF(d) = Σ 1 / (k + rank_i(d)).
  • Assign ranks starting at 1; a missing document contributes nothing for that list.
  • Deduplicate by a stable document identifier before calculating the final order.
  • Fetch more candidates from each retriever than the number of results you plan to return.

Make fusion observable in a Talqora retrieval pipeline

Talqora uses regional S3 Vectors for dense search and Quickwit BM25 for sparse search. Treat their result lists as separate retrieval signals and record enough metadata to explain the fused ranking: source ranks, whether a document appeared in both lists, and its final fusion score. This turns ranking review from guesswork into an inspectable engineering process.

Evaluate with query sets that reflect production behavior rather than only broad natural-language questions. Include exact product names, internal identifiers, abbreviations, misspellings, multi-concept questions, and paraphrased requests. Review not only whether the desired document appears, but whether it is high enough in the fused list for the next stage of the application.

  • Log dense rank, BM25 rank, and fused rank for returned documents.
  • Track overlap between candidate lists; low overlap can be informative rather than erroneous.
  • Inspect failures by query type before changing k or candidate depths.
  • Version the embedding model, sparse analyzer configuration, and fusion policy together.