AI Engineering Glossary
Plain-language definitions for the terms that come up most when building production AI and LLM systems: RAG, embeddings, agents, MCP, fine-tuning, evaluation, caching, prompt injection, and EU AI Act compliance, each linked to a deeper guide.
Foundations
- Large Language Model (LLM)
- A neural network, typically a transformer, trained on huge amounts of text to predict the next token. Everything else in this glossary is built on top of that one capability.
- Generative AI
- The broader category of models that produce new content, text, images, audio, code, rather than classifying or scoring existing input. LLMs are the dominant form of generative AI in software today.
- Token / Tokenization
- The unit an LLM actually reads and bills by, roughly a few characters or a word fragment, not a word. Pricing, context limits, and latency are all expressed in tokens. Read the full guide on tokens →
- Context Window
- The maximum number of tokens a model can read in a single request. A bigger window doesn't mean every token in it gets used equally well. Read why longer context windows hurt accuracy →
- System Prompt
- The instructions set once, outside the user-visible conversation, that define a model's role, constraints, and tone for every turn that follows.
- Inference
- The act of running a trained model to produce an output, as opposed to training it. Inference cost and latency are what users actually experience.
- Hallucination
- A confident, fluent output that is factually wrong or unsupported by any source. It gets worse, not better, when a model is fine-tuned on facts it never learned during pre-training. See why fine-tuning can increase hallucination →
- Frontier Model
- The most capable general-purpose model a lab currently offers, as opposed to its smaller and cheaper siblings. No single frontier model wins on every axis, so the practical question is which one to use for which workload. Compare Claude, GPT, and Gemini →
Retrieval & Knowledge
- Embedding
- A vector representation of text, image, or other data such that semantically similar inputs end up close together in vector space. It is what makes "search by meaning" possible. Read the full guide on embeddings →
- Vector Database
- A database built to store embeddings and find the nearest ones to a query vector quickly, at scale, under filtering constraints. Pinecone, Qdrant, Weaviate, Milvus, and pgvector are common choices. Compare vector databases →
- Retrieval-Augmented Generation (RAG)
- A pattern where relevant text is retrieved from an external source and injected into a model's prompt at request time, so it can answer using knowledge it wasn't trained on. Read what nobody tells you about RAG in production →
- Chunking
- Splitting source documents into smaller pieces before embedding them, so retrieval can return focused, relevant text instead of entire documents. The chunk is the unit of retrieval, which makes it one of RAG's most consequential decisions. Read the full guide on chunking strategies →
- Fine-Tuning
- Continuing a model's training on your own examples so its weights, and therefore its behavior, change. It is for changing how a model acts, not for teaching it new facts. Read how to choose between RAG, fine-tuning, and prompting →
- Prompt Engineering
- Designing instructions, examples, and structure to get more out of a model's existing capability, without touching its weights or adding external data. Read why prompting is the cheapest rung on the ladder →
Agents & Orchestration
- AI Agent
- An LLM given instructions, tools, and enough autonomy to decide which actions to take to complete a task, rather than just answering a single prompt. Read about building agents with the OpenAI Agents SDK →
- Multi-Agent System
- An architecture where several specialized agents, each with a narrow role, collaborate on a task instead of one agent handling everything itself. Read Agent Skills vs Multi-Agent Systems →
- Agent Skills
- Reusable packages of procedural knowledge, instructions, scripts, and conventions, that an agent loads on demand to perform a task well, as an alternative to splitting that knowledge across multiple agents. Read the full comparison →
- Tool Use / Function Calling
- The mechanism by which a model emits a structured request to invoke a function with specific arguments, rather than only generating text. See how this relates to MCP →
- Model Context Protocol (MCP)
- An open protocol, introduced by Anthropic, that standardizes how AI applications connect to external tools, data, and prompt templates, so an integration can be built once and reused across applications. Read the full explainer on MCP →
- Handoff
- A mechanism that lets one agent delegate a task to another, more specialized agent mid-conversation, a core primitive in frameworks like the OpenAI Agents SDK. Read more about handoffs →
Evaluation & Reliability
- LLM Evaluation
- The practice of systematically measuring whether an LLM application's outputs are actually good, with a representative dataset and consistent metrics, rather than judging from a handful of demo prompts. Read why your demo works but production fails →
- LLM-as-a-Judge
- Using a separate LLM call to score or grade the output of another LLM call against a rubric, useful for evaluating qualities that are hard to check with a deterministic rule. Read how to use it without fooling yourself →
- Golden Dataset
- A curated set of inputs with known-correct expected outputs, used as the stable benchmark you re-run every time you change a prompt, a model, or a pipeline. Read how to build one →
- Lost in the Middle
- The well-documented tendency of LLMs to pay less attention to information placed in the middle of a long input, even when nothing in the prompt has technically exceeded the context limit. Read the full breakdown →
- Pilot-to-Production Gap
- The distance between AI agent projects that get built and those that actually ship. Independent 2026 studies put the share of enterprise agent pilots that never reach production at 86 to 89 percent, and the causes are mostly structural rather than technical. Read where agent pilots actually die →
- Compounding Error
- The way per-step reliability multiplies across an agent chain. A step that succeeds 95 percent of the time completes a twenty-step task only about 36 percent of the time, which is why reducing step count usually beats improving individual steps. Read the reliability math →
Security
- Prompt Injection
- An attack where text an LLM reads, whether typed directly or hidden in a document, email, or web page, contains instructions that hijack the model's behavior. There is no patch that removes the vulnerability entirely. Read the full guide on prompt injection →
- Lethal Trifecta
- Simon Willison's framing for when an agent becomes a serious data-theft risk: access to private data, exposure to untrusted content, and the ability to communicate externally, all at once. Read the full breakdown →
- Jailbreak
- A prompt crafted to make a model ignore its own safety instructions or stated rules, distinct from prompt injection, which hijacks a model using content it processes rather than instructions a user writes directly.
- Guardrails
- Checks, often other model calls or deterministic rules, that validate inputs or outputs before they reach a user or a sensitive action, used to contain what an agent is allowed to do. Read how guardrails fit into the Agents SDK →
Efficiency & Cost
- Quantization
- Reducing the numerical precision a model's weights are stored and computed in, which shrinks memory use and speeds up inference at a small, often negligible, accuracy cost. Read the full guide on model optimization →
- Pruning
- Removing weights or structures from a trained model that contribute little to its output, producing a smaller model with similar accuracy. Read more about pruning →
- Knowledge Distillation
- Training a smaller "student" model to mimic the outputs of a larger "teacher" model, producing a cheaper model that keeps most of the teacher's capability on the tasks that matter. Read more about distillation →
- Prompt Caching
- An infrastructure-level optimization where a provider reuses the processing it already did on an identical prompt prefix, so you pay a reduced rate for the part of your input that has not changed. It can never return a wrong answer, because the model still generates the response. Read the full guide on caching →
- Semantic Caching
- An application-level shortcut that returns a stored answer when a new question is semantically similar enough to one already answered, skipping the model call entirely. Far riskier than prompt caching, because "similar enough" is a bet. Read about the failure modes →
- Latency
- The time between sending a request to a model and receiving the full response, driven heavily by how many tokens go in and come out.
Regulation & Compliance
- EU AI Act
- The European Union regulation governing AI systems, applying in stages between 2025 and 2028 and classifying systems by risk. It binds anyone placing an AI system on the EU market or serving EU users, regardless of where the company is based. Read what developers have to ship →
- Article 50 (Transparency Obligations)
- The part of the AI Act that requires people to be told when they are interacting with an AI system, and requires synthetic content to be marked and labelled. Enforceable from 2 August 2026, and the part that touches the largest number of ordinary software products. Read the Article 50 breakdown →
- Provider vs Deployer
- The two roles the AI Act assigns different duties to. A provider develops a system and places it on the market under its own name; a deployer uses one under its own authority. Wrapping a foundation model and shipping it under your brand usually makes you a provider, not a deployer. Work out which role you are in →
- High-Risk AI System
- An AI system in a use case the AI Act treats as carrying significant risk to health, safety, or fundamental rights, such as recruitment, credit, education, or essential services. These obligations were deferred to December 2027 for stand-alone systems. See the current timeline →
- Machine-Readable Marking
- Embedding a signal in generated content that software can detect as artificially generated, typically signed provenance metadata such as C2PA Content Credentials plus an imperceptible watermark. A visible caption alone does not satisfy the requirement. Read how to actually mark content →
- C2PA / Content Credentials
- An open provenance standard, now ISO/IEC 21694, that attaches a cryptographically signed manifest describing how a piece of media was created and edited. Widely adopted, but routinely stripped by platforms, screenshots, and re-encoding. See where it fits in a marking pipeline →
- Deepfake
- AI-generated or AI-manipulated image, audio, or video content resembling real people, places, or events. Under the AI Act, the organisation publishing it has to disclose that it is artificial, with a softer rule for evidently artistic or satirical work. Read the disclosure rules →
- General-Purpose AI (GPAI) Model
- A model trained broadly enough to be used across many different tasks, which the AI Act regulates separately from the systems built on top of it. Obligations sit on the model provider, and have been in force since August 2025. See how this splits from your own obligations →