Weigh the tradeoffs

Hybrid search improves coverage but runs two retrieval paths and complicates tuning. Dense search handles synonyms but may miss rare codes; lexical search nails codes but misses paraphrases. Reranking improves precision at added latency and cost. This is why “best” must always finish the sentence: best for which users, traffic, risk, hardware, budget, and deadline?

Start with constraints, not preferences. A hard privacy rule, an accessibility requirement, or a two-second interaction budget eliminates designs before a weighted score is useful. Among feasible choices, compare expected utility. A simple decision model is:

utility = task_value - error_cost - inference_cost - delay_cost - operations_cost

The terms need not share natural units; agreed weights make assumptions visible. Run sensitivity analysis. If a small change in the error-cost weight flips the winner, the decision is fragile and needs better evidence or a reversible rollout.

For a technical support search assistant, compare at least a simple baseline, a moderate design, and a maximal design. Rate each on quality, severe failures, latency, variable cost, privacy, debuggability, and team burden. Do not let one average score compensate for a prohibited failure. Apply gates first.

For “router XG-410 red status after update,” lexical search ranks the exact model manual first but misses the paraphrased symptom article; dense search finds the article but ranks an XG-400 page above it. RRF merges ranks, then a cross-encoder puts the XG-410 firmware article first. recall@5 rises from 72% and 81% alone to 93% hybrid. The lesson is not the final setting; it is the sequence of evidence and the willingness to choose a less impressive configuration when it better satisfies the whole system.

Count retries, fallbacks, duplicated tokens, review labor, index updates, GPU idle capacity, incident response, and evaluation maintenance. Per-token price alone often reverses the wrong decision. Likewise, local operation is not free after hardware purchase, and automation is not free when humans must repair low-quality cases.

Opportunity cost matters too. A complex architecture may gain two quality points while delaying the feedback loop by a month. A simpler version with a clean trace and rollback can teach more. Choose the smallest design that tests the riskiest assumption.

Prefer reversible choices under uncertainty: canary traffic, versioned indexes, expiring memory, adapters rather than irreversible data changes, and feature flags around orchestration. Restrict early exposure to cases where failure is recoverable. Consequences—not model size—determine the required approval level.

Finally, state who bears each cost. A system can improve an aggregate metric while shifting work to reviewers, slowing users on poor connections, or degrading one language. Segment results and ask whether the people receiving benefits are also absorbing the errors. That question turns an abstract tradeoff into an accountable product decision.