Cosine, dot product, and distance

Cosine similarity is the dot product divided by the product of the vectors’ lengths:

cos(q, x) = (q · x) / (||q|| ||x||)

Its usual range is -1 to 1, although practical text embeddings may occupy a narrower range. A score near 1 means the vectors point in similar directions, 0 means they are perpendicular, and -1 means opposite directions. These geometric descriptions do not translate automatically into “same meaning,” “unrelated,” and “opposite meaning.” The model’s training determines what directions represent.

For q = [1, 0] and a = [2, 0], the dot product is 2, the lengths are 1 and 2, and cosine similarity is 2 / (1 × 2) = 1. For b = [0.8, 0.6], its length is 1, so cosine similarity is 0.8. Cosine ranks a first because it ignores the difference in length.

Cosine is common for semantic text retrieval because direction often captures the useful signal while vector magnitude is not intended to affect relevance. That is a model-specific convention, not a law. Use the metric recommended for the model and validated on your data.

The dot product multiplies corresponding coordinates and adds them:

q · x = q1x1 + q2x2 + ... + qn xn

For the warm-up vectors, q · a = 2 and q · b = 0.8, so dot product also ranks a first. Now compare c = [10, 1] and d = [1, 0] against q. The raw dot score for c is 10, even though c is not perfectly aligned; d scores 1 despite perfect alignment. Magnitude can dominate.

If all vectors are normalized to length one, dot product equals cosine similarity. That fact explains why some vector indexes offer “inner product” search for normalized embeddings. Do not assume a library normalizes automatically. Check whether vectors are normalized by the model, client, database, or not at all, and apply the same processing to stored items and queries.

Dot product can be the correct choice when a model was trained and evaluated for it. Magnitude may encode useful properties under that training objective. Replacing it with cosine because cosine sounds semantic may change rankings.

Euclidean distance is the straight-line distance between points:

distance(q, x) = sqrt((q1 - x1)^2 + ... + (qn - xn)^2)

Smaller means closer. For q = [1, 0] and a = [2, 0], the distance is 1. For b = [0.8, 0.6], it is sqrt(0.2^2 + (-0.6)^2) = sqrt(0.40) ≈ 0.632. Euclidean distance therefore ranks b ahead of a, unlike cosine.

On normalized vectors, squared Euclidean distance and cosine similarity produce the same ordering because ||q - x||^2 = 2 - 2 cos(q, x) when both lengths are one. The numeric values differ, but the nearest neighbor is the same. This equivalence disappears when lengths vary.

Euclidean distance is intuitive for spaces where absolute position and scale matter. It can work for embeddings designed for it. As with other metrics, the name alone does not determine suitability.

A cosine score of 0.82 is not “82% relevant” and does not mean the answer is 82% correct. Scores are not calibrated probabilities. Their distribution varies by model, data, query length, language, and content type. A threshold copied from another project may reject good results or admit bad ones.

Build a labeled validation set. For each query, mark which candidates are relevant, partially relevant, hard negatives, or irrelevant. Examine score distributions and ranking metrics. If the system must choose “match” versus “no match,” choose a threshold according to the cost of false positives and false negatives, then validate it on held-out examples.

Ranking and thresholding are distinct. A query can always have a top neighbor even when every candidate is poor. Consider a recipe query sent to an index containing only tax forms. The highest-scoring tax form remains top one. A minimum score, an out-of-domain detector, reranking, or a “no useful result” policy can prevent the application from treating relative best as genuinely good.

Never compare raw scores across different models as though they share a scale. A score of 0.7 from model A may represent a different percentile and quality than 0.7 from model B.

Embedding similarity is strong at paraphrase and conceptual matching, but exact terms matter for product codes, names, dates, error messages, and legal clauses. Hybrid search combines vector similarity with lexical search such as BM25. Metadata filters enforce hard constraints. A reranker can inspect the query and each candidate together with more precision than independent embeddings.

Imagine a query for error code E1047 reset procedure. Semantic search may retrieve general reset instructions, while keyword search finds the exact code. A hybrid score or reranker can put the code-specific page first. Conversely, keyword-only search may miss “display remains black” when the article says “screen will not illuminate”; embeddings recover the paraphrase.

Production ranking may therefore be a pipeline: filter by access and product, retrieve 50 vector and keyword candidates, merge them, rerank the top 20, and return five. Evaluate the complete pipeline. Improving nearest-neighbor speed is not useful if the final results become less relevant.

Start with the model’s documented metric. Confirm normalization behavior. Create numerical unit tests with known vectors so an accidental switch from similarity to distance does not reverse ordering. Then evaluate representative search judgments.

Also inspect latency, index support, and approximation settings. An approximate vector index may trade a little recall for large speed gains. That approximation is separate from the embedding metric: cosine can be searched exactly or approximately.

When reporting results, name the model, vector processing, metric, index configuration, candidate count, filters, and evaluation set. “We use cosine” is not enough to reproduce or assess the system.