Why one retrieval signal is not enough

Dense search represents text as vectors and is useful when query and document wording differ but their meaning is related. A query such as "reset access after changing phones" can retrieve documentation titled "recover your account" even when the terms do not closely overlap.

Sparse retrieval with BM25 depends on lexical evidence. That makes it especially valuable for exact strings such as error messages, API field names, internal acronyms, version labels, SKUs, and identifiers. In these cases, semantic similarity alone can blur distinctions that exact matching preserves.

  • Dense retrieval helps with paraphrases and concept-level matches.
  • BM25 helps when rare or exact terms carry the intent.
  • Neither signal should be assumed to dominate every query type.

Build separate candidate pools before fusion

For each query, retrieve a bounded ranked list from the dense index and another from the BM25 index. Talqora’s architecture pairs regional S3 Vectors for dense search with Quickwit BM25 for sparse search, giving an application two distinct retrieval signals to work with.

Keep the two retrieval calls logically independent. Each can return document IDs, ranks, and any metadata needed to filter or hydrate results. Use the same corpus identifiers across both indexes so that a document appearing in both lists can be recognized and merged deterministically.

  • Apply tenant, access-control, language, and document-status filters consistently to both paths.
  • Start with equal candidate depths, such as the top N results from each path, then adjust using relevance evaluations.
  • Deduplicate by stable document or chunk ID rather than by text content.
  • Store source-specific rank information for debugging and offline analysis.

Fuse ranks without comparing incompatible scores

Dense similarity values and BM25 scores are not naturally interchangeable. Their scales can vary with embedding choice, query length, corpus composition, and sparse-index behavior. Directly adding raw scores can make one retriever dominate for accidental numerical reasons rather than relevance.

Reciprocal rank fusion (RRF) avoids that problem by combining ranks instead of raw scores. For a document d, calculate RRF(d) = Σ 1 / (k + rank_i(d)) across retrieval lists where the document appears. The constant k softens the advantage of a single first-place result; choose and validate it with a representative relevance set rather than assuming one value is universal.

After fusion, return the highest-ranked unique candidates to a reranker, an answer-generation step, or a user-facing results page. Log which retrieval path contributed each result. When a result is surprising, that provenance makes it easier to determine whether the issue is chunking, indexing, filtering, query construction, or fusion.

  • Use rank-based fusion when score scales differ.
  • Evaluate semantic, exact-match, mixed, and no-result queries separately.
  • Inspect failures where BM25 retrieves the right identifier but dense search does not, and vice versa.
  • Treat fusion settings as versioned retrieval configuration.