Foundations to Advanced Systems: LLMs for Product Managers
What Is an Embedding?
3.3 What Is an Embedding?
Embeddings are the engine that makes RAG work. You do not need to understand the mathematics. But you do need a mental model precise enough to make good product decisions.
Imagine a map where every piece of text has coordinates, and texts that mean similar things are located near each other. Google and Search engine are close together but Google and Retail are far apart. Refund policy and how do I return a product are neighbors.
Vector embeddings turn unstructured data like text into numerical representations in a high-dimensional space, allowing AI models to understand semantic relationships between different pieces of data. When a user's query is embedded, the system finds the document chunks whose coordinates are closest to the query's coordinates. This is semantic search. It matches by meaning, not just keywords. A user who asks "how do I get my money back" will retrieve chunks about refund policies even if those chunks never use the phrase "get my money back."
Why Embedding Quality Is a Product Decision
Not all embedding models are equal, and choosing the wrong one has direct product consequences. A general-purpose embedding model handles everyday language well. But in specialized domains, legal, medical, financial, technical, it often FAILS to capture the nuanced meanings that matter most. Generalist embedding models usually will not capture the semantic nuances of domain-specific data, causing the system to struggle to prioritize the right chunks for retrieval.
Get the retrieval right, and the generation takes care of itself. Get it wrong, and no amount of prompt engineering will save you.
For product managers, the practical takeaway is this: the embedding model is not an infrastructure decision that belongs only to engineering. It is a product quality decision. The right embedding model for your domain meaningfully improves the relevance of retrieved content, which directly improves the quality of everything the model generates from that content.