Why raw dense and BM25 scores should not be mixed

A dense retrieval score reflects the similarity function, embedding model, and vector index configuration used for the query. A BM25 score reflects term frequency, document frequency, field length, and analyzer behavior. Even when both scores are larger for apparently better matches, their numeric ranges and distributions do not have a shared meaning.

This matters for a Talqora deployment that uses regional S3 Vectors for dense search and Quickwit BM25 for sparse search. A rule such as “0.7 times the vector score plus 0.3 times the BM25 score” can change behavior unexpectedly when embeddings, text fields, analyzers, or corpus composition change. The weighting may look precise while relying on incomparable inputs.

  • Dense retrieval can recover paraphrases and semantically related language.
  • BM25 can reward rare, exact, and lexical matches such as product codes or quoted terms.
  • Score ranges can vary by query, index, and retrieval configuration.
  • A stable merge should avoid assuming that one system’s score unit equals another’s.

Fuse two candidate lists with Reciprocal Rank Fusion

RRF starts by issuing the same user query to both retrieval paths. Each path returns an ordered list of document identifiers. For every document, add a contribution based on its rank in each list: 1 divided by k plus the rank. The final score is the sum of those contributions, and documents are sorted by that total.

The constant k reduces the gap between adjacent top positions and prevents a single first-place result from overwhelming all other evidence. A commonly used starting value is 60, but it is a tuning parameter rather than a universal truth. Keep the retrieval depth and k explicit in configuration so changes are reviewable and testable.

  • Retrieve a candidate list from S3 Vectors and a candidate list from Quickwit BM25.
  • Assign ranks beginning at 1 within each list.
  • For each document d, compute RRF(d) = Σ 1 / (k + rank(d)).
  • Treat a document missing from a list as contributing zero from that retrieval path.

Implement the merge as a small, observable retrieval stage

Use a canonical document identifier across dense and sparse indexing. After both searches return, de-duplicate by that identifier, calculate RRF scores, and retain the top fused candidates for the next stage of the application. If document metadata is returned by both paths, select one authoritative source or define deterministic field precedence rather than letting response order decide.

Evaluate the fused results with a query set that includes both semantic and lexical cases. For example, include natural-language questions, abbreviation-heavy queries, identifier lookups, and queries where an exact title matters. Log which source contributed to each fused result; this makes it easier to diagnose whether a weak outcome came from candidate generation, text analysis, embeddings, or the fusion rule.

  • Choose the same candidate depth for initial experiments, then tune depths separately if needed.
  • Record dense rank, BM25 rank, and final fused rank for inspected queries.
  • Test duplicate handling when multiple chunks map to one parent document.
  • Version the embedding model, BM25 field configuration, k value, and fusion logic together.