Why independent retrieval paths matter

A dense retriever represents the meaning of a query and document in an embedding space. This is valuable for questions such as “how do I rotate credentials?” when relevant documentation uses wording like “replace an access token.” The match can succeed even without substantial term overlap.

Sparse BM25 retrieval scores term occurrence and document statistics. It is especially important for queries containing model numbers, API fields, customer-facing product names, log messages, or quoted fragments. In these cases, semantic similarity can be helpful, but exact lexical evidence should not be discarded.

  • Use dense retrieval to broaden semantic recall.
  • Use BM25 to retain exact-match and rare-term behavior.
  • Run both paths against the same logical document corpus.
  • Keep document IDs stable across indexes so results can be merged.

Fuse ranks, not raw scores

Dense similarity scores and BM25 scores are not naturally comparable. Their scales depend on the embedding method, index configuration, query length, corpus composition, and scoring implementation. Adding raw scores can give one retriever accidental dominance simply because its numeric range is larger.

A practical alternative is reciprocal rank fusion (RRF). For each result, assign a contribution based on its rank in each list, then sum the contributions. A common form is score(d) = Σ 1 / (k + rank_i(d)), where rank_i is the one-based rank of document d in result list i and k is a constant chosen by the application. The method works from ordering alone and rewards documents that appear near the top of either or both lists.

  • Fetch a candidate list from dense search and another from BM25.
  • Deduplicate candidates by the shared document ID.
  • Compute fusion from ranks rather than incomparable raw scores.
  • Return the fused top results, optionally retaining per-source ranks for debugging.

Make fusion observable and query-aware

Start with one stable fusion policy before introducing many query-specific rules. Log the query, candidate IDs, dense rank, BM25 rank, fused rank, and the final selected results. These records make it possible to investigate failures: a missing result may indicate indexing coverage, while a low-ranked result may indicate ranking behavior.

Then evaluate with a representative set of real query classes. Include natural-language questions, identifier lookups, quoted error strings, short keyword searches, and queries with abbreviations. If identifier-heavy queries repeatedly need lexical precision, retrieve more BM25 candidates or apply a carefully tested query classifier that adjusts candidate depth or fusion weights.

  • Create relevance judgments for queries that reflect production traffic.
  • Inspect failures by query type, not only aggregate relevance metrics.
  • Version embedding, chunking, and fusion-policy changes together.
  • Treat metadata filters as part of retrieval correctness when documents have scope or access boundaries.