Mastery: connect the pieces

A complete explanation of hybrid search for RAG now has four connected claims. Keyword and vector search are two different librarians. One recognizes exact shelf labels such as “ERR_CONN_204”; the other understands paraphrases such as “connection closes after login.” Hybrid search asks both and merges their candidates. A lexical engine scores term overlap, often with BM25, while dense retrieval compares query and document embeddings. Scores are not directly comparable, so systems normalize or combine ranked positions with reciprocal-rank fusion. A reranker can then inspect the merged candidates more deeply. Hybrid retrieval merges different candidate signals; reranking scores a small candidate set more accurately; generation writes from selected evidence. Metadata filtering enforces scope and should happen consistently, not be treated as another relevance hint.

Turn those claims into a design for a technical support search assistant. State the user job, data boundary, uncertain model contribution, deterministic controls, evaluation set, release gate, production signal, and failure response. If any item is missing, the concept is not yet operational.

Use this spoken diagram:

authorized input -> scoped evidence -> learned operation
                 -> deterministic validation -> bounded action
                 -> outcome + trace -> evaluation and improvement

At every arrow ask: what representation crosses, who owns it, what can be lost, and how is it versioned? Tune lexical and dense candidate counts, fusion weights or RRF constant, field boosts, metadata filters, query rewriting, reranker depth, final top-k, and duplicate removal. Preserve exact identifiers and quoted phrases rather than rewriting them away. The controls should be few enough to understand and complete enough to constrain the severe failures.

Hybrid search improves coverage but runs two retrieval paths and complicates tuning. Dense search handles synonyms but may miss rare codes; lexical search nails codes but misses paraphrases. Reranking improves precision at added latency and cost. Choose one tradeoff and defend it quantitatively. Name a hard constraint, a primary metric, and the cost you accept. Then name evidence that would reverse your decision. This last step protects the design from becoming identity or vendor loyalty.

A defensible statement sounds like: “We choose configuration B because it passes the privacy and severe-error gates, improves task success on the target slice, and stays within the p95 latency budget. We will reconsider if traffic or review cost crosses the recorded threshold.”

Combining raw BM25 and cosine scores gives one engine accidental dominance; filters differ between paths; duplicate chunks fill top-k; query expansion deletes product codes; stale indexes disagree; and a reranker trained on web passages mishandles logs or tables. Pick the most consequential failure and walk through trigger, earliest signal, containment, owner, recovery, and prevention. Build judged query-document pairs covering exact codes, paraphrases, mixed queries, and no-answer cases. Compare recall@k, MRR or nDCG, duplicate rate, filter correctness, latency, and end-to-end citation support. Ablate lexical-only, dense-only, fusion, and reranking. Monitoring should reuse the evaluation construct where possible, while acknowledging that production labels may arrive late.

Answer without notes:

1. What does this concept change: evidence, learned behavior, runtime state, coordination, or measurement?
2. Which neighboring concept is commonly confused with it?
3. Which intermediate artifact would you inspect first?
4. Which knob has the largest quality/resource interaction?
5. What hard gate cannot be traded for average quality?
6. What baseline could disprove the need for the complex design?
7. How would you detect harm hidden by an aggregate metric?
8. What is the safe state during uncertainty?

Now explain the worked evidence: For “router XG-410 red status after update,” lexical search ranks the exact model manual first but misses the paraphrased symptom article; dense search finds the article but ranks an XG-400 page above it. RRF merges ranks, then a cross-encoder puts the XG-410 firmware article first. recall@5 rises from 72% and 81% alone to 93% hybrid. If you can identify the causal chain, calculate the consequential change, propose an alternative hypothesis, and choose a reversible response, you have moved from vocabulary to engineering judgment.

End with one page containing: context, alternatives, assumptions, case-set version, configuration IDs, metric table, gates, selected option, rejected options, owner, rollout, rollback, and review date. This artifact makes future disagreement productive because teammates can challenge evidence or weights instead of reconstructing hidden reasoning.