Start with two candidate sets, not one universal query

A dense query begins with an embedding of the user’s text. It is useful when the user describes an idea in language that differs from the wording in the indexed document. For example, a search for “reduce login friction” may still be relevant to content discussing session renewal or authentication flows.

A sparse BM25 query works directly with terms in the query and indexed text. It is especially valuable when exact token overlap carries meaning: product SKUs, function names, ticket IDs, database fields, quoted phrases, and error messages. Rather than choosing a single winner globally, retrieve a bounded candidate set from each path.

  • Use the same document identifiers across dense and sparse indexes.
  • Keep metadata needed for filtering available to both retrieval paths.
  • Choose a candidate depth that leaves room for overlap and unique results from each method.
  • Log the source path for every candidate so ranking behavior can be inspected later.

Merge rankings with reciprocal rank fusion

A practical first merge policy is reciprocal rank fusion (RRF). Instead of comparing raw dense similarity scores with BM25 scores, which are not inherently on the same scale, RRF rewards documents that rank well in one or both lists. This avoids requiring score normalization before the initial hybrid experiment.

For each document d, calculate an aggregate score such as RRF(d) = Σ 1 / (k + rank_i(d)), where rank_i is the document’s position in retrieval list i and k is a fixed constant. Sort by the aggregate score, deduplicate by document ID, and return the top results. The exact constant is a tuning choice, but keeping it fixed while evaluating changes makes comparisons easier.

  • Treat missing documents from a list as contributing no score from that list.
  • Deduplicate before presenting results, even when the same content is indexed through multiple paths.
  • Keep the per-source rank alongside the fused score for debugging.
  • Apply business or permission filters before returning the final ranked set.

Evaluate by query shape, then refine deliberately

Hybrid retrieval should be evaluated against a query set that reflects real traffic rather than a single aggregate relevance number. Segment queries into categories such as exact lookup, troubleshooting, broad conceptual research, acronym-heavy queries, and short ambiguous queries. These categories reveal where dense or sparse retrieval contributes unique value.

Inspect cases where the two lists disagree. If sparse search misses a known identifier, investigate tokenization, field selection, or document text. If dense search returns broad thematic neighbors instead of the intended answer, inspect chunk boundaries, embedding input construction, and whether the query needs metadata constraints. The goal is to make failure modes observable before adding complexity.

  • Create judged examples with one or more acceptable documents per query.
  • Measure candidate recall before focusing on final answer quality.
  • Review zero-result and low-click queries separately.
  • Version retrieval settings and record which dense and sparse configurations produced each result set.