Why combine dense and sparse retrieval?

Dense retrieval represents text as embeddings and retrieves items that are near a query in vector space. This can be valuable when a user’s wording differs from the wording in the source material, such as a query for “resetting account access” matching documentation titled “credential recovery.”

Sparse retrieval with BM25 scores documents from term occurrences and term-frequency statistics. It is often a strong signal for product names, error codes, proper nouns, quoted phrases, and other vocabulary where exact lexical overlap is meaningful. Neither signal should be treated as universally sufficient.

  • Dense retrieval helps with paraphrases and semantic relatedness.
  • BM25 helps preserve exact-match behavior for terms and identifiers.
  • Independent result lists provide useful debugging evidence when relevance is poor.
  • A fusion layer can combine signals without assuming their raw scores share a common scale.

Fuse ranks, not incompatible scores

Raw retrieval scores are usually not directly comparable across systems. A cosine-like vector similarity and a BM25 score have different ranges, distributions, and meanings. Adding them together without calibration can make ranking sensitive to score scale rather than relevance.

Reciprocal rank fusion avoids that issue by using each document’s position in a ranked list. For every document returned by either retriever, calculate an RRF score as the sum of 1 divided by k plus its rank in each list. Documents appearing near the top of multiple lists rise naturally.

  • RRF score: score(d) = Σ 1 / (k + rank_i(d)).
  • Choose a positive k to reduce the difference between adjacent top ranks.
  • Use only the top N candidates from each retriever to bound request and merge work.
  • Treat a document absent from a list as contributing zero from that retriever.

Implement the pattern around Talqora retrieval calls

In a Talqora-based application, send the query embedding to the dense retrieval path backed by regional S3 Vectors, and send the original query text to the sparse retrieval path backed by Quickwit BM25. Keep document identifiers stable across both indexes so results can be joined before fusion.

Log the dense rank, sparse rank, and final fused rank for a sampled set of queries. These fields make it easier to identify whether an issue is caused by embedding generation, lexical matching, stale indexing, or fusion behavior. Start with a small evaluation set containing semantic queries, exact-name queries, and identifier-heavy queries before changing retrieval parameters.

  • Use the same canonical document ID in dense and sparse indexes.
  • Retrieve separate candidate lists, then deduplicate by document ID.
  • Apply filters consistently to both retrieval paths when your application requires them.
  • Evaluate fused results against representative queries before tuning candidate counts or k.