Why raw score mixing is fragile

Dense and sparse systems typically produce scores with different meanings. A dense-search similarity value depends on the embedding model and distance or similarity function. A BM25 score depends on term frequency, document frequency, field configuration, and query terms.

Adding or averaging those values assumes they are calibrated to the same scale, which is often untrue. Even normalizing scores per query can be unstable: a single unusually high result or a short result list can materially change the normalized values.

  • Do not assume a BM25 score of 8 is comparable to a dense similarity score of 0.8.
  • Score distributions can change after an embedding-model, analyzer, or corpus update.
  • Rank positions are usually easier to interpret consistently than raw scores.

Fuse ranked lists with RRF

Retrieve a candidate list from dense search and another from BM25, then identify documents by a stable document ID. For each occurrence of a document at rank r in a list, add 1 / (k + r) to its fused score. Sum the contributions across lists and sort by the total.

The constant k reduces the gap between adjacent top ranks and prevents one rank-1 placement from overwhelming all other evidence. A commonly used starting point is k = 60, but it is a tuning parameter rather than a universal rule. Evaluate it against representative queries and relevance judgments when available.

  • Use rank 1 for the first result in each source list.
  • Deduplicate by document ID before returning results.
  • Keep each source's rank and contribution for debugging.
  • Apply filters consistently to dense and sparse retrieval before fusion.

Build an observable retrieval pipeline

In a Talqora-oriented architecture, an application can request dense candidates from regional S3 Vectors and sparse candidates from Quickwit BM25, then perform RRF in its retrieval layer. This keeps fusion logic explicit and makes it possible to evolve the merge policy without treating either backend's score as a shared unit.

Log more than the final ordering. For sampled queries, record the query, filters, candidate counts, source ranks, fused score, and selected document IDs. These records help diagnose cases where exact-match documents disappear, semantic matches dominate unexpectedly, or filtering creates thin candidate sets.

  • Start with equal treatment of dense and BM25 lists before introducing weights.
  • Choose candidate depths large enough for overlap and complementary results to appear.
  • Test identifier-heavy, natural-language, short, and ambiguous queries separately.
  • Version retrieval settings such as candidate depth, k, embedding model, and BM25 configuration.