Why candidate union is a useful default

A single retrieval method can omit relevant documents before any downstream logic has a chance to inspect them. Dense retrieval may underweight an exact string that is central to the query. BM25 may miss a relevant document when its language differs substantially from the user's wording.

Candidate union reduces dependence on either method's first-stage recall. Request the top k dense results and the top k sparse results, deduplicate by a stable document or chunk identifier, and pass the resulting set to a ranking step. The ranking step can be a simple rule at first; the important design choice is preserving candidates from both retrieval signals.

  • Dense retrieval contributes semantic matches.
  • BM25 contributes lexical and exact-term matches.
  • Deduplication prevents a document found by both paths from appearing twice.
  • A bounded k keeps downstream ranking work predictable.

Choose candidate sizes from query intent, not guesswork

Using the same k for every query is simple, but query types often have different needs. A query containing a long identifier, a quoted phrase, a version number, or an error code is a strong signal that sparse retrieval deserves meaningful representation in the candidate set. A natural-language question with few distinctive tokens may benefit from a larger dense candidate set.

Start with conservative, explicit limits and log how many unique candidates each source contributes. If dense and sparse results overlap heavily, increasing both limits may add little value. If one source frequently contributes unique documents that are later selected by users or evaluators, that source may need a larger candidate budget for that query class.

  • Detect exact-match signals such as quoted text, IDs, codes, and versions.
  • Record dense count, sparse count, overlap count, and final union size.
  • Set a maximum union size before any more expensive ranking step.
  • Review candidate composition by query category, not only overall averages.

Make the merge policy observable and easy to revise

The union itself is straightforward; the harder part is understanding why a result was returned. Store source provenance with every candidate: dense only, sparse only, or both. This makes debugging concrete when a result looks surprising, and it avoids treating retrieval as an opaque single score.

After unioning candidates, use a ranking policy appropriate to the application. A minimal policy can prioritize documents returned by both sources and retain source-specific candidates. More advanced policies can use calibrated scores, metadata filters, freshness rules, or a separate ranking model. Keep retrieval, merging, and final ranking as distinct stages so each can change without silently changing the others.

  • Attach provenance labels to every merged candidate.
  • Apply metadata and authorization filters consistently across both paths.
  • Log the final rank alongside the candidate's retrieval sources.
  • Test changes with representative semantic, exact-term, and mixed queries.