Start with two independent candidate lists

A practical hybrid retrieval pipeline begins by asking the same query of both retrieval systems. The dense path produces candidates based on vector similarity, and the sparse path produces candidates based on BM25 term matching. Keep the initial result lists independent: each signal should have the opportunity to contribute documents the other path would miss.

This separation matters because the two methods fail differently. A query containing a product code, error string, or uncommon proper noun may depend heavily on lexical overlap. A natural-language question with varied wording may benefit from semantic similarity. Independent retrieval preserves those complementary strengths before any ranking decision is made.

  • Run dense and sparse retrieval against the same logical document corpus.
  • Request more than the final number of results from each path.
  • Keep document identifiers stable across both indexes.
  • Record which retrieval path contributed each candidate.

Fuse ranks instead of comparing raw scores

Dense similarity scores and BM25 scores are not naturally interchangeable. Their ranges, distributions, and meanings can vary with indexing choices, query composition, and corpus changes. Adding raw scores without calibration can unintentionally make one retrieval path dominate for reasons unrelated to result quality.

A rank-based fusion rule avoids requiring direct score equivalence. For each document, assign value according to its position in each result list, then combine those values. Reciprocal rank fusion is one simple example: a document receives a contribution from every list in which it appears, with higher-ranked appearances contributing more.

  • Deduplicate candidates by document identifier before producing final results.
  • Use ranks as the common unit when score scales are not calibrated.
  • Choose a candidate depth for each path and revisit it as the corpus evolves.
  • Retain per-path rank metadata for debugging and evaluation.

Use query intent to guide, not replace, fusion

Some query patterns provide useful routing hints. Quoted text, exact identifiers, filenames, and error messages often indicate that sparse retrieval deserves attention. Broader questions, paraphrases, and conceptual prompts may benefit from dense retrieval. These hints can influence candidate depth or weighting, but they should not become hard assumptions without evaluation.

The operational discipline is to inspect disagreements. When dense search finds a result absent from BM25, ask whether it is a helpful semantic match or a topic-adjacent distraction. When BM25 wins on an exact phrase, check whether that precision is valuable to the user. A small set of representative queries can reveal where fusion rules need adjustment.

  • Create an evaluation set containing exact-match and conceptual queries.
  • Include queries with acronyms, IDs, dates, and uncommon terminology.
  • Review fused results alongside the separate dense and sparse lists.
  • Change one fusion variable at a time so ranking shifts remain explainable.