Start with two independent retrieval paths

Treat dense and sparse retrieval as separate candidate generators. For a query, dense search can retrieve records whose embeddings are nearby, while sparse BM25 retrieval can retrieve records sharing important query terms. Each path produces a ranked list with its own score scale and ranking behavior.

In Talqora’s architecture, dense search is backed by regional S3 Vectors and sparse search is backed by Quickwit BM25. Keeping the paths conceptually separate is useful because it avoids forcing a semantic similarity score and a BM25 score into a direct comparison before their meaning has been established.

  • Request a bounded top-k candidate list from dense search.
  • Request a bounded top-k candidate list from BM25 search.
  • Apply the same required filters to both paths.
  • Keep source-specific ranks and scores for debugging.

Fuse ranks before trying to fuse raw scores

Raw dense and BM25 scores are not automatically comparable. Their ranges can vary with the query, corpus, analyzer behavior, embedding model, and retrieval implementation. Adding them directly can make one path dominate for reasons unrelated to result quality.

A practical baseline is rank-based fusion. Reciprocal rank fusion, for example, assigns each result credit based on its position in each ranked list, then sums that credit across lists. A document returned by both retrieval paths receives support from both semantic and lexical evidence, while a strong result from only one path can still appear.

  • Deduplicate candidates by a stable document or chunk identifier.
  • Store each candidate’s dense rank and sparse rank.
  • Use a fixed fusion rule before introducing learned weighting.
  • Inspect queries where one retrieval path contributes all final results.

Evaluate contribution, not just the final list

A hybrid system should be evaluated with query sets that reflect real retrieval tasks. Include exact-name searches, error messages, product terminology, paraphrased questions, and queries containing both a specific term and a broader intent. These cases reveal whether dense and sparse retrieval are contributing complementary candidates.

Operationally, log which path retrieved each final candidate and whether that candidate was found by both. This makes it easier to diagnose regressions: a change in embeddings may affect dense-only results, while tokenization or indexing changes may affect BM25-only results. The final ranking is important, but the origin of its candidates is often the faster debugging signal.

  • Create a small labeled query set before tuning fusion behavior.
  • Measure whether relevant items are retrieved by dense, sparse, or both paths.
  • Review misses where relevant records appear in neither candidate set.
  • Tune candidate depths separately from the final number of returned results.