Treat dense and sparse retrieval as candidate generators

Dense retrieval is useful when a query and a document express similar meaning with different vocabulary. A query for “reset account credentials,” for example, may still retrieve content written as “recover access to your profile.” The embedding path is designed to capture this semantic neighborhood.

BM25 is useful when exact wording matters. Product names, error codes, identifiers, uncommon acronyms, and quoted phrases can be decisive signals that semantic similarity may soften. Quickwit BM25 provides a sparse retrieval path for these lexical matches.

  • Use dense retrieval to discover semantically related documents.
  • Use BM25 to preserve exact-term and rare-token matches.
  • Do not assume one candidate source can reliably cover the other source’s failure modes.

Allocate a candidate budget to each path before fusion

Instead of asking each retriever for an arbitrary number of results, define explicit candidate budgets. For example, retrieve a dense candidate set from regional S3 Vectors and a separate BM25 candidate set from Quickwit, then combine their document identifiers into one deduplicated pool. The budget is a recall-control setting, not a final ranking decision.

Separate budgets also make debugging clearer. If a relevant document never appears in either candidate set, the issue is candidate generation. If it appears in the pool but finishes too low, the issue is fusion or downstream ranking. Those are different problems and should be measured separately.

  • Start with independent dense and BM25 candidate counts.
  • Deduplicate documents after collecting both result sets.
  • Record which retrieval path contributed each candidate.
  • Tune candidate counts using representative queries, especially exact-match and paraphrase-heavy cases.

Fuse with rank-aware signals, then inspect query classes

Scores from dense retrieval and BM25 are not automatically comparable. They come from different models and scales, so directly adding raw scores can create unstable behavior. A rank-aware approach, such as reciprocal rank fusion, avoids assuming that a dense score and a BM25 score mean the same thing.

After fusion, review results by query class rather than relying on an overall impression. Identifier-heavy support queries, short navigational queries, natural-language questions, and multi-concept research queries can each need different candidate budgets. The goal is not to make dense and sparse retrieval identical; it is to let each contribute where it is strongest.

  • Prefer a fusion method that does not require raw-score equivalence.
  • Inspect whether relevant documents came from dense, BM25, or both paths.
  • Create query groups based on lexical specificity and semantic ambiguity.
  • Adjust budgets only after confirming whether misses occur before or after fusion.