Why raw dense and BM25 scores should not be added directly

Dense retrieval and BM25 produce scores for different mathematical reasons. A vector similarity score reflects the relationship between embeddings, while BM25 is driven by term frequency, document frequency, and document length. Even when both searches return the same document, the magnitude and distribution of their scores can differ substantially.

Adding those values together assumes that a one-point change in one system means the same thing as a one-point change in the other. That assumption is usually fragile across query types, corpus changes, embedding models, and BM25 configuration. Rank-based fusion avoids this calibration problem by using the ordering each retriever provides.

  • Use dense retrieval for semantic phrasing, paraphrases, and conceptually related content.
  • Use BM25 for exact names, error codes, product identifiers, and rare terms.
  • Treat each retriever's score as local to that retriever unless it has been explicitly calibrated.

Fuse two ranked lists with Reciprocal Rank Fusion

Run a dense query against the vector index and a sparse BM25 query against the text index. For every document returned by either list, assign an RRF contribution based on its rank: 1 divided by k plus rank. Sum that contribution across lists in which the document appears.

The formula is RRF(d) = Σ 1 / (k + rank_i(d)). The constant k dampens the difference between adjacent ranks. A commonly used starting value is 60, but it is a tuning parameter rather than a universal rule. The important property is that documents appearing near the top of either list receive meaningful credit, while documents supported by both lists accumulate more credit.

  • Request a candidate list from both dense and sparse retrieval paths.
  • Normalize document identity before fusion so the same record is merged correctly.
  • Use one-based ranks: the first result has rank 1.
  • Sort candidate documents by their summed RRF score and return the top results.

Implement the fusion layer as a small, observable application step

Because Talqora is API-first, an application can keep retrieval orchestration explicit: issue the dense and sparse requests, collect ranked identifiers, and fuse them before presenting results or sending a smaller candidate set to a later reranker. This keeps the retrieval policy visible in application code rather than hiding assumptions in score arithmetic.

Evaluate the fused output with a representative query set. Include semantic questions, exact-identifier lookups, short queries, and queries containing both a concept and a specific term. Inspect failures by recording each result's dense rank, sparse rank, and final fused rank. That trace makes it easier to determine whether a miss comes from candidate generation, document text, embeddings, or fusion settings.

  • Start with equal contribution from dense and sparse ranked lists.
  • Keep enough candidates from each path that useful overlap can emerge.
  • Log per-query ranks and the final fusion score for relevance debugging.
  • Change k or list depths only after reviewing labeled or human-judged results.