Why raw dense and sparse scores are difficult to add

A dense-search score is produced by a vector similarity function and depends on details such as embedding normalization, query distribution, and index configuration. A BM25 score is based on term frequency, document frequency, and length normalization. Even when both are useful rankings, a score of 0.8 from one system has no inherent relationship to a score of 0.8 from the other.

This becomes visible with mixed queries. For a query containing an exact ticket number plus a broad natural-language request, BM25 may strongly rank the document with the identifier, while dense retrieval may rank conceptually related documents. Adding raw scores can accidentally let one ranking dominate simply because its numerical range is wider.

Before tuning weighted score addition, verify that score values are calibrated for the same query population and collection. If they are not, rank-based fusion is a safer baseline because it requires only ordered candidate lists.

  • Dense and BM25 scoring functions express different signals.
  • Score ranges can shift across queries and corpus updates.
  • Exact tokens and semantic paraphrases often benefit from different retrievers.
  • Rank positions are easier to compare than uncalibrated raw scores.

Fuse candidate lists with reciprocal rank fusion

RRF assigns each document a contribution based on its position in each result list. For a document d, the fused score is the sum of 1 divided by k plus its rank in every list where it appears. The constant k reduces the difference between very high ranks and lower ranks, preventing a single rank-one placement from overwhelming all other evidence.

In a Talqora application, retrieve a bounded candidate list from dense search backed by regional S3 Vectors and another from Quickwit BM25. Join results by a stable document or chunk identifier, calculate the RRF score in the application, then return the merged ordering. A document absent from one list simply receives no contribution from that retriever.

Start with the same candidate depth for both branches so that one source is not silently given more opportunities to contribute. For example, request the top N dense and top N BM25 candidates, deduplicate by chunk ID, and fuse the union.

  • Formula: RRF(d) = Σ 1 / (k + rankᵢ(d)).
  • Use one-based ranks: the first result has rank 1.
  • Deduplicate documents before presenting final results.
  • Keep retrieval filters consistent across dense and sparse branches.

Evaluate fusion with queries that expose retrieval trade-offs

A useful evaluation set should include more than broad topical questions. Include exact identifiers, product names, abbreviations, quoted phrases, paraphrased questions, and queries containing both a precise token and a conceptual intent. Label the passages that are acceptable answers, not only a single preferred document.

Compare dense-only, BM25-only, and RRF results at the depth users actually see. Inspect failures manually: if an identifier-bearing passage disappears, check tokenization and filtering; if paraphrased intent disappears, inspect chunking and embedding inputs. Fusion is not a substitute for correct data preparation, but it can make the two retrieval signals complementary.

Once RRF is a stable baseline, make changes one at a time. Candidate depth, the RRF constant, chunk boundaries, metadata filters, and query rewriting can each change relevance. Recording the query, branch ranks, and final fused rank makes those changes explainable.

  • Build a test set with lexical, semantic, and mixed-intent queries.
  • Measure each branch separately before judging the fused result.
  • Log branch rank and fused rank for every returned candidate.
  • Tune candidate depth and k using held-out relevance judgments.