AI & LLM Glossary

A

Agent — An LLM looped with tools (search, APIs, code execution) that plans and acts toward a goal without a human in every step. Not every chatbot is an agent; the bar is autonomous multi-step execution. → AI Agents Explained

Attention mechanism — The part of a transformer that lets each token "look at" other tokens in the sequence to build context-aware representations. It's why transformers scale to long contexts and why "attention is all you need" became a meme. → Transformer Architecture Explained

API rate limit — The cap on how many requests or tokens you can send to a model API per minute. Hitting it in production means queued users and 429 errors — design for backoff and batching from day one.

B

Batch inference — Running many inputs through a model in a single API call instead of one-by-one. Cheaper per token and higher throughput, but adds latency variance when one slow item blocks the batch.

BM25 — A classic keyword search algorithm that ranks documents by term frequency and inverse document frequency. Still beats pure embeddings on exact-match queries like SKUs, error codes, and API method names. → Reranking & Hybrid Search

C

Chain-of-thought (CoT) — Prompting the model to show its reasoning steps before the final answer. Improves accuracy on math and logic tasks at the cost of more output tokens.

Chunking — Splitting documents into smaller passages before embedding them for retrieval. Bad chunking is the #1 cause of bad RAG — not the model. → Chunking Strategies for RAG

Context window — The maximum number of tokens a model can process in one request (input + output). Bigger windows cost more; stuffing everything in context is not always better than retrieval.

Context engineering — Designing what goes into the model's context — system prompt, retrieved chunks, tool results, conversation history — to maximize answer quality within token limits. → Context Engineering Explained

Corpus — The full set of documents you're indexing, searching, or training on. In RAG, corpus quality and freshness matter more than model choice.

D

Decoder — The half of a transformer that generates output tokens one at a time, using attention over prior tokens. GPT-style models are decoder-only; T5-style models use both encoder and decoder.

Distillation — Training a smaller "student" model to mimic a larger "teacher" model's outputs. Cuts inference cost and latency while preserving most of the quality.

E

Embedding — A dense vector representation of text (or images) that captures semantic meaning. Similar meanings → nearby vectors. The foundation of semantic search and RAG. → Embeddings Explained

Evals — Structured tests that measure model or system quality — retrieval recall, answer accuracy, latency, cost. Without evals you're guessing whether a change helped. → Evals Explained

F

Few-shot learning — Giving the model a handful of input→output examples in the prompt so it infers the pattern without weight updates. Cheap and fast; quality degrades past ~5–10 examples.

Fine-tuning — Updating a model's weights on your data so it learns a style, format, or behavior — not facts you can paste into a prompt. Use when prompting can't get the tone or structure right. → Fine-Tuning Explained

Function calling — The model outputs structured JSON describing which tool to invoke and with what arguments, instead of free-form text. The backbone of tool-using agents. → Function Calling Explained

G

Generative AI — AI systems that create new content — text, images, code, audio — rather than only classifying or predicting. LLMs are the dominant generative AI modality in 2026. → What is Generative AI?

Grounding — Tying model outputs to verifiable sources — retrieved docs, database rows, tool results — so answers aren't pure hallucination. RAG is the most common grounding technique.

Guardrails — Input/output filters and validators that block harmful content, PII leaks, off-topic responses, or malformed JSON before users see them. → Guardrails & Output Validation

H

Hallucination — When a model states false information confidently, often because it wasn't grounded in real data. Mitigate with retrieval, citations, and refusal prompts — not bigger models alone. → AI Hallucinations Explained

Hybrid search — Combining dense vector search with sparse keyword search (BM25), then merging or reranking results. Standard practice for production RAG on mixed corpora. → Reranking & Hybrid Search

I

Inference — Running a trained model to produce outputs. Distinct from training; inference cost and latency dominate production economics.

In-context learning (ICL) — The model adapting its behavior based on examples and instructions in the prompt, without gradient updates. Prompt engineering and few-shot prompting are forms of ICL.

K

KV cache — Stored key/value tensors from prior tokens so the model doesn't recompute attention for the full prefix on every new token. Critical for fast streaming inference.

L

LLM (Large Language Model) — A neural network trained on vast text to predict the next token, scaled large enough to follow instructions, reason, and use tools. The default building block for AI products in 2026. → What is an LLM?

Latency — Time from request to first token (TTFT) or full response. Users feel anything over ~2s for chat; batch jobs tolerate more.

LoRA (Low-Rank Adaptation) — A parameter-efficient fine-tuning method that trains small adapter matrices instead of the full model. Cheaper and faster than full fine-tuning for most use cases.

M

MCP (Model Context Protocol) — An open standard for connecting LLMs to external tools and data sources through a consistent server interface. Think USB-C for AI integrations. → MCP Explained

Multimodal — Models that accept and/or produce more than one modality — text, images, audio, video. Vision + language is the most common multimodal combo in production. → Multimodal AI Explained

P

Parameters — The learned weights inside a neural network. "7B parameters" means 7 billion numbers the model adjusts during training; more parameters generally mean more capability and more cost.

Post-training — Everything after pre-training — instruction tuning, RLHF, safety fine-tuning — that makes a base model useful in chat. This is where "helpful assistant" behavior comes from. → How AI Models Are Trained

Pre-training — The initial phase where a model learns language structure by predicting tokens on massive unlabeled text. Expensive, done once per model generation.

Prompt engineering — Crafting system prompts, examples, and context to steer model behavior without changing weights. The highest-ROI skill for most AI products. → Prompt Engineering Basics

Prompt injection — An attack where user input tricks the model into ignoring its system prompt or leaking secrets. Mitigate with input sanitization, output validation, and least-privilege tool access.

Q

Quantization — Reducing model weight precision (e.g. FP16 → INT8 → INT4) to shrink memory footprint and speed up inference, with a small quality trade-off.

R

RAG (Retrieval-Augmented Generation) — Retrieve relevant document chunks at query time, paste them into the prompt, then generate an answer grounded in those sources. The default pattern for private-data Q&A. → RAG Explained

Reranking — A second-pass model that re-scores retrieved chunks for relevance before sending them to the LLM. Cheap insurance against bad top-k retrieval. → Reranking & Hybrid Search

RLHF (Reinforcement Learning from Human Feedback) — Training the model to prefer responses humans rated highly, using a reward model and reinforcement learning. How chat models learn to be helpful and harmless. → How AI Models Are Trained

S

Semantic search — Finding documents by meaning (via embeddings) rather than exact keyword match. Great for paraphrased questions; weak on SKUs and error codes without hybrid search.

Streaming — Sending model output token-by-token as it's generated instead of waiting for the full response. Improves perceived latency even when total time is unchanged.

Structured outputs — Constraining the model to return valid JSON (or another schema) via API features or post-validation. Essential for tool pipelines and UI rendering. → Structured Outputs Explained

System prompt — The hidden instruction block that sets the model's role, rules, and tone for a session. Most "AI personality" and safety rules live here.

T

Temperature — A sampling parameter controlling randomness. 0 = deterministic (same input → same output); higher values = more creative, less reliable for factual tasks.

Token — The atomic unit models process — roughly ¾ of an English word. Pricing, context limits, and latency are all measured in tokens.

Tool use — Letting the model call external functions — search APIs, databases, code runners — during a conversation. Implemented via function calling or MCP. → Function Calling Explained

Top-k / Top-p (nucleus sampling) — Decoding strategies that limit which tokens the model can pick next. Top-k keeps the k most likely tokens; top-p keeps the smallest set whose cumulative probability exceeds p.

Transformer — The neural architecture behind modern LLMs, built on self-attention and parallelizable training. Replaced RNNs as the default for language tasks. → Transformer Architecture Explained

V

Vector database — A database optimized for storing embeddings and running similarity search at scale. pgvector, Qdrant, and Pinecone are common choices. → Vector Databases Explained

Z

Zero-shot — Asking the model to perform a task with no examples in the prompt — only instructions. Works for well-known tasks; fails on niche formats without few-shot examples.