Treat dense and sparse retrieval as complementary candidate generators

Dense retrieval represents documents and queries as vectors, making it useful when a query and a relevant document express the same idea with different language. Sparse retrieval with BM25 emphasizes overlapping terms and term rarity, which is valuable for model names, error codes, product SKUs, legal phrases, and other exact vocabulary.

Rather than requiring either path to be universally correct, use each to produce a bounded candidate set. The combined set gives downstream ranking logic a chance to recover documents that one retrieval method would not have surfaced on its own.

  • Use dense retrieval for semantic paraphrases and concept-level matches.
  • Use BM25 for exact terminology, uncommon tokens, and query wording that must be preserved.
  • Keep candidate limits explicit so latency and downstream work remain predictable.

Fuse ranks before trusting raw scores

Dense similarity scores and BM25 scores are not naturally comparable. They come from different scoring systems, and a numerical value from one path should not be assumed to have the same meaning as a value from the other. Directly adding uncalibrated scores can make one retriever dominate for accidental numerical reasons.

A simple starting point is rank-based fusion. For each document returned by either path, assign a contribution based on its position in each ranked list, then sum the contributions. Reciprocal-rank-style methods are attractive because they reward documents that appear near the top without requiring score scales to match.

  • Deduplicate documents by a stable document identifier before final ranking.
  • Record each document's dense rank, sparse rank, and fused rank.
  • Use missing-rank handling consistently when a document appears in only one candidate set.
  • Consider score calibration only after collecting query-specific evaluation data.

Make hybrid behavior debuggable from the first release

Hybrid retrieval is easier to improve when every result can be explained. Log the query, retrieval configuration, candidate counts, source ranks, and the final fusion inputs. These traces reveal whether poor results came from candidate generation, fusion, document preparation, or a later ranking stage.

Evaluate with query groups instead of one aggregate number alone. A support corpus may have identifier-heavy queries, natural-language questions, and short ambiguous searches; each group can favor a different balance of dense and sparse retrieval. Reviewing representative failures is often more useful than tuning on a small set of memorable examples.

  • Create a small labeled set containing semantic, exact-match, and mixed-intent queries.
  • Inspect queries where only one retrieval path found the relevant document.
  • Version embeddings, document chunking rules, and fusion settings together.
  • Use regional S3 Vectors for dense search and Quickwit BM25 for sparse search as distinct observable paths in the retrieval design.