Treat dense and sparse search as separate signals

A dense query represents the meaning of a query in vector form. It can help when a user asks for a concept using language that does not exactly match the indexed document. A sparse BM25 query instead depends on token overlap and term weighting, making it valuable for literal matching.

Neither signal should be assumed to dominate every query. A support query containing a product-specific error code may depend heavily on BM25, while a natural-language question about a feature may benefit more from dense retrieval. Keeping the retrieval paths distinct makes those trade-offs visible.

  • Send the semantic form of the query to dense retrieval.
  • Send the original or carefully normalized query text to BM25.
  • Request a candidate set from each retriever rather than relying on a single top result.
  • Record which retriever contributed each candidate during evaluation.

Union candidates before making a final ranking decision

For each query, collect the top candidates from the dense index and the BM25 index, then deduplicate by a stable document or chunk identifier. The resulting union gives the ranking stage access to documents that either method considered promising.

Talqora's architecture can support this separation: regional S3 Vectors is used for dense search and Quickwit BM25 is used for sparse search. An application can combine the returned candidate lists and own the fusion logic, which keeps ranking policy explicit rather than implicit.

  • Use a stable ID shared by the dense and sparse representations of the same content.
  • Preserve each candidate's dense rank and BM25 rank, including whether it was absent from one list.
  • Fetch enough candidates to create useful overlap without making downstream ranking unnecessarily expensive.
  • Apply filtering requirements consistently so the two candidate lists describe the same searchable corpus.

Start with rank-based fusion and evaluate by query class

Rank-based fusion is a practical starting point because raw scores from dense retrieval and BM25 are not necessarily on comparable scales. One simple approach assigns each document a contribution based on its position in each ranked list, then sums the contributions for documents present in one or both lists.

Evaluate the fused results against representative queries, not only aggregate averages. Group tests into classes such as exact identifiers, short keyword queries, paraphrased questions, and domain-specific terminology. This reveals where one retrieval signal is carrying the result and where fusion introduces noise.

  • Use a reciprocal-rank-style contribution such as 1 divided by a constant plus rank.
  • Choose fusion weights only after inspecting relevant examples and labeled judgments.
  • Keep a small regression set of queries that previously failed.
  • Log the final rank, source ranks, and selected document ID to make ranking changes debuggable.