Every LLM carries two kinds of knowledge. Understanding the difference between them is what allows you to make smart decisions about when a model is enough on its own, and when it needs something more.

What the Model Already Knows: Parametric Knowledge

When an LLM answers a question about the French Revolution or explains how a binary search works, it is drawing entirely from what it learned during training. This is called parametric knowledge, because it lives in the model's parameters, the billions of numerical weights that were shaped by processing enormous amounts of text.

Think of it as everything a person knows from memory. It is intrinsic, compressed, and immediately accessible. The model cannot update itself. LLMs' parametric knowledge is static, frozen at the point when training ended, with no awareness of anything that happened after that moment. For many use cases, this is fine. Writing assistance, general reasoning, explaining concepts, none of these require up-to-the-minute information. But for products that need to work with your data, your policies, or your customers, static knowledge is a serious constraint.

What Lives Outside the Model: Non-Parametric Knowledge

Non-parametric knowledge is everything stored outside the model in an external system, a database, a document store, a knowledge base. Unlike parametric knowledge, it is:

Updatable: Add a new document today, it will found in your search.
Auditable: You can inspect exactly what information was used to generate a response
Specific: It can contain proprietary information that was never and could never be part of any public training dataset

RAG combines a neural language model for text generation (parametric memory) with a retrieval system that fetches relevant information from an external database (non-parametric memory). This allows RAG to leverage the fluency of language models while enriching responses with up-to-date and verifiable information.

Foundations to Advanced Systems: LLMs for Product Managers

Module 1Introduction to Large Language Models in Product Context

Module 2The LLM System Architecture

Module 3Advanced LLM Workflows

Module 4Evaluation

Module 5LLM Risks

Module 6LLM Productions

Module 7Strategic Applications for Product Managers

The Core Limitation: Parametric vs External Knowledge