Text embeddings and cosine similarity

Give each sentence a list of numbers

An embedding is a vector: an ordered list of numbers that represents an item. For text, a trained model produces that list from a word, sentence, or passage. The geometry of these vectors can capture useful relationships for a task. Google's introduction to embedding spaces.

Picture a map of help articles. Password-reset instructions and login troubleshooting might sit near one another. A recipe for banana bread should be somewhere else. The map is an analogy for comparing those representations.

The model learns its representation during training. You do not usually assign one coordinate to "password" and another to "bread." Real embeddings have many dimensions. Individual coordinates need not have an obvious human label.

Start with a small, deliberately invented map

The sketch above uses three hand-written, two-dimensional vectors to illustrate the comparison. The coordinates are invented for teaching, and the scores come from those coordinates. Your browser does not run an embedding model.

Our query is [1, 0.1]. A password-reset answer points in almost the same direction, while a recipe points elsewhere. Select a different answer to see its direction and similarity score.

This toy map makes the arithmetic visible. With a real model, the numbers come from encoding text. A visualization in two dimensions simplifies the full space.

Compare directions with cosine similarity

Cosine similarity divides the dot product of two vectors by the product of their lengths. Its value ranges from −1 to 1 for nonzero vectors. A value near 1 means the vectors point in almost the same direction. The choice of similarity measure depends on the model and task. Sentence Transformers documents its similarity measures.

function cosine(a, b) {
  const dot = a.reduce((sum, value, i) => sum + value * b[i], 0);
  const length = Math.hypot(...a) * Math.hypot(...b);
  return length === 0 ? 0 : dot / length;
}

const query = [1, 0.1];
const answers = [
  { text: 'Reset your password', vector: [0.92, 0.2] },
  { text: 'Troubleshoot signing in', vector: [0.72, 0.43] },
  { text: 'Bake banana bread', vector: [-0.1, 1] },
];

const ranked = answers
  .map(answer => ({
    text: answer.text,
    score: cosine(query, answer.vector),
  }))
  .sort((a, b) => b.score - a.score);

console.table(ranked);

This example assumes the vectors have the same dimensions. Its zero-length check returns zero as a fallback. Mathematically, cosine similarity is undefined for a zero vector. The reset answer should rank first and the recipe last.

Turn the comparison into a search

For a real experiment, encode a small set of support articles with a sentence-embedding model. Encode a question with the same model. Calculate the similarities. Return the closest candidates. Sentence Transformers shows text encoding and comparison.

Try ten short questions before building a larger system. Include alternate wording, spelling mistakes, and an unrelated question. Record which answer appears first. Read that answer to decide whether it helps.

Keep the question list as a repeatable check. If you change the model or split articles into smaller passages, run the same questions again. This lets you compare results across changes instead of relying on one impressive result.

A high score still needs interpretation

"I can sign in" and "I cannot sign in" discuss almost the same subject while describing different states. Include pairs like these in your experiments. They help distinguish topic similarity from the exact relationship your application needs.

Similarity scores depend on the representation and comparison method. Even familiar cosine scores can behave unexpectedly across models and training choices. Research on the limits of cosine similarity explains why the score needs context.

Retrieval gives you candidates to inspect. In a support search, a good candidate should also match the product, version, and situation of the user. Ranking alone does not make that decision.