Failures, bias, and mismatch

Embeddings often place texts together because they discuss the same subject, even when their claims conflict. “The valve must remain open,” “the valve must remain closed,” and “Is the valve open?” share vocabulary and context. Negation, quantities, dates, and direction can be underweighted relative to broad topic.

Numbers are especially risky. A policy allowing 30 days may be close to one allowing 60 days. Version 2.1 may be near 2.10. A medication passage for 5 mg may resemble one for 50 mg. Preserve exact text, filter structured fields, and use reranking or deterministic checks when these distinctions drive action.

Adversarial or merely repetitive text can also attract retrieval. A document stuffed with common query terms may rank highly despite low value. Embeddings are not security boundaries. Enforce access controls outside similarity search, validate source ingestion, and treat retrieved content as untrusted input when it reaches a generative model.

A general embedding model may know common language but poorly represent specialized meanings. “Cell” means something different in biology, telecommunications, spreadsheets, and prisons. In law, “consideration” has a technical meaning. In manufacturing, an internal part code may have almost no public-language context.

Domain mismatch appears as plausible but shallow neighbors. The model retrieves broad pages instead of exact procedures, confuses abbreviations, or clusters products by marketing language rather than engineering function. Benchmarking on generic sentence pairs will not reveal these failures.

Build an evaluation set from the target domain, including jargon, abbreviations, identifiers, and hard negatives. Compare a general model, a domain-trained model, hybrid lexical–vector search, and reranking. Fine-tuning may help when enough high-quality relevance pairs exist, but it adds training, governance, and versioning work. Often better chunking, metadata, and hybrid search deliver the first improvement.

“Multilingual” covers many levels of performance. A model may align English and Spanish well but handle a low-resource language poorly. It may represent formal writing better than regional speech, transliteration, or code-switching. Translation can improve retrieval in some systems while losing names, legal meanings, or cultural context.

Evaluate each supported language and direction. Cross-language search—an Arabic query retrieving an English document—is different from same-language Arabic search. Include native speakers in relevance judgments. Machine-translated test queries can make evaluation unrealistically tidy because they preserve source-language structure.

Locale also matters inside one language. “Pants,” “public school,” and date formats vary across regions. Product availability, law, and support procedures may differ even when sentences are semantically close. Use locale metadata as a hard constraint where applicable.

A safe fallback may be language-specific lexical search, translation with disclosure, or escalation rather than pretending equal quality. Report coverage gaps so users and operators know where the system is less reliable.

Embedding spaces can reflect social stereotypes and unequal representation in training data. Names, occupations, dialects, neighborhoods, disabilities, or demographic terms may acquire associations that affect retrieval and ranking. Even without explicit demographic fields, text can contain proxies.

Bias is application-dependent. A stereotyped association in creative similarity search has different consequences from one used to rank job candidates, patients, or fraud reports. For high-impact decisions, embedding similarity should not become an unexplained decision score. Legal and domain experts may need to determine whether the use is appropriate at all.

Measure outcomes across relevant groups and language varieties, while respecting privacy and the limits of demographic labels. Examine exposure: whose documents are retrieved, whose queries receive no result, and which groups experience false matches. Average Recall@10 can conceal a severe gap if the majority group dominates the test set.

Mitigation may involve changing sources, model, objective, filters, ranking rules, user experience, or the decision itself. Removing a few “biased dimensions” is not a dependable general fix because information is distributed across the space.

Many failures come from wiring rather than model capability:

• old and new model vectors are mixed in one collection;
• documents are normalized but queries are not;
• the index uses Euclidean distance while evaluation assumed cosine;
• query and passage modes are reversed;
• deleted documents remain searchable;
• permission metadata is missing;
• text is truncated before the answer;
• an approximate index is configured for too little recall.

These defects often return reasonable-looking results, which makes them hard to detect. Record model and pipeline versions, add unit tests with known neighbors, monitor ingestion counts, and run a fixed retrieval suite during deployment.

Drift matters after launch. New products, slang, policies, or customer behavior can move the workload away from the evaluation set even though the embedding model is unchanged. Sample live failures, refresh judgments, and define rollback triggers.

When a result is wrong, ask where the failure began:

1. Coverage: Does a correct source exist and is it current?
2. Parsing: Was the relevant text extracted?
3. Chunking: Does one retrievable unit contain enough context?
4. Representation: Does the model connect the query and passage?
5. Retrieval: Did the index return the neighbor under correct filters?
6. Ranking: Did fusion or reranking demote it?
7. Presentation: Did the interface or generator misuse good evidence?

This sequence prevents “change the model” from becoming the answer to every issue. If the source was absent, no embedding model could retrieve it. If access filtering removed it correctly, higher similarity should not override permissions.

Keep difficult cases as regression tests. Include contradictions, near-identical numbers, rare dialect, ambiguous abbreviations, stale versions, and out-of-domain queries. A useful evaluation set is not a random sample of easy successes; it represents the mistakes the system must avoid.