AI Vocabulary - Tokens, Embeddings and Everything Else

Spend any time around AI in 2026 and you hit a wall of vocabulary - tokens, embeddings, parameters, quantization, RAG, MCP - dropped into release notes and API docs with no definitions attached. This is the reference for that wall. Each term below gets a short, plain definition, grouped by where you actually run into it instead of alphabetically, so you can find the one word that's blocking you.

We have already done the big picture in this series: what AI, ML, generative AI, and LLMs actually mean, and how an LLM turns your text into an answer. For the handful of terms those posts go deep on, you'll get a one-line definition here and a link to the full explanation there.

Words for how a model reads your text

Before a model can answer, it converts your text into something it can do math on.

Token. The unit a model reads. Not a character, not quite a word - usually a word-piece. "unbelievable" might be three tokens, "the" is one. Everything you send is chopped into tokens first, and each becomes an integer ID. Rough rule for English: one token is about four characters, or three-quarters of a word. The mechanics of what happens next are in the how-LLMs-work post.

Tokenizer. The component that does the chopping. It learns its rules from training data using byte-pair encoding: start with characters, then repeatedly merge the most common pairs into single tokens. Common words become one token; rare words get split. Google's SentencePiece is a widely used implementation.

Context window. The maximum number of tokens a model can hold at once, counting both your input and its output. In 2026, 128K tokens is ordinary and some models advertise a million or more. Go past it and the oldest tokens drop out of view - the model does not remember your conversation, it re-reads whatever still fits on every turn.

Embedding. A list of numbers - a vector - that represents meaning. Text with similar meaning lands at nearby points in that number space, which is how a machine treats "car" and "automobile" as close. The how-LLMs-work post shows where embeddings sit in the pipeline.

Vector database. Once text is embeddings, you need somewhere to store millions of them and a fast way to ask "which of these is closest to this one?". That is a vector database, and it powers semantic search - matching on meaning instead of exact keywords. pgvector, an extension for Postgres, is a common starting point.

Words that describe the model itself

These show up whenever people compare models.

Parameters (weights). The numbers a model learned during training - its "knowledge", spread across billions of dials. "7B" means seven billion parameters, "70B" means seventy billion. Do not confuse this with tokens: parameters are the fixed size of the model, tokens are the flowing text it processes. More parameters usually means more capability and more hardware to run it.

Base model vs instruct model. A base model is a raw next-token predictor - it continues your text but was never taught to be helpful. An instruct (or chat) model is a base model given extra training to follow instructions and hold a conversation. When you use a chatbot, you are using the instruct version.

Foundation model / frontier model. A foundation model is trained broadly enough to be adapted to many tasks rather than built for one. A frontier model is shorthand for the current most-capable models at the leading edge. Both are loose, marketing-adjacent terms, but that is what people mean.

Open weights vs open source. Not the same, and the difference matters. Open weights means you can download the trained parameters and run them yourself - many live on Hugging Face - but the training data and code may stay private. Open source, used strictly, means code, data, and weights are all released under an open license. Most "open" models you hear about are open-weights, not fully open source.

Quantization. Compressing a model by storing its weights at lower precision - 16 bits down to 8, or 4, sometimes fewer. A 4-bit version is roughly a quarter of the size and runs on far cheaper hardware, for a small and usually acceptable quality hit. This is why you can run a capable model on a laptop now; the GGUF format used by llama.cpp and Ollama made it routine. Quantization shrinks the weights, not the context window.

Distillation. Training a small "student" model to imitate a big "teacher" model, getting much of the teacher's quality at a fraction of the size. A lot of the surprisingly-good small models are distilled.

Words for how it generates an answer

The model does not compose a whole reply and hand it over. It produces one token at a time, and these words control that.

Inference. Running a trained model to get output. This is the step you pay for and wait on every request. Training is the expensive one-time process that created the model; inference is everything after, with the parameters frozen.

Temperature, top-p, top-k. Three knobs for how random the next-token pick is:

• Temperature controls overall randomness. Low (0.1-0.2) makes the model pick the safe, obvious token; high (1.0+) flattens the odds and makes it wander. The how-LLMs-work post shows exactly what it does to the numbers.
• Top-p (nucleus sampling) considers only the smallest group of tokens whose probabilities add up to p, then picks from those.
• Top-k is blunter: consider only the k most likely tokens, ignore the rest.

Max tokens. The ceiling you set on how long the output can be, in tokens. Hit it and the response is cut off mid-sentence - a very common cause of "why did it stop?".

Latency, throughput, TTFT, tokens/sec. Speed words. Latency is how long one request takes end to end. Throughput is total work a server pushes across all users. TTFT (time to first token) is how long you wait before the first word appears. Tokens per second is the streaming speed after that; a specialized inference provider like Groq competes on making that number big.

Words for getting it to do what you want

This is where most day-to-day work happens.

Prompt, system prompt, user prompt. A prompt is whatever you feed the model. The system prompt is the standing instruction that sets its role and rules ("you are a terse code reviewer"); the user prompt is the actual question. The system prompt is why the same model can behave like a lawyer in one app and a pirate in another.

Prompt engineering. Deliberately shaping the input to get a better output - phrasing, ordering, giving examples, being explicit about the format you want. Less mystical than it sounds; mostly clear instructions and testing.

Zero-shot, few-shot, in-context learning. Zero-shot: you ask with no examples. Few-shot: you paste a handful of examples into the prompt so the model catches the pattern. Learning from examples in the prompt without any retraining is called in-context learning, and it is one of the properties that made these models feel like a step change - the GPT-3 paper named it.

Chain-of-thought. Getting the model to work through the steps before committing to an answer instead of blurting a conclusion. Asking it to "think step by step" measurably improves hard reasoning, which the original paper showed; by 2026 the "reasoning" models do this internally before they show you anything.

Fine-tuning. Taking an existing model and training it further on your own narrower dataset so it specializes - your tone, your domain, your format. You change the weights a little, unlike prompting, which changes nothing.

RAG (retrieval-augmented generation). Instead of hoping the model memorized a fact, you fetch relevant documents at query time - usually with embeddings and a vector database - and paste them into the prompt so it answers from real, current sources. It is the standard fix for "the model does not know about my data", and it cuts down on invented answers. The original RAG paper is from 2020; the idea is now everywhere.

Grounding. Tying an answer to verifiable source material rather than the model's own memory. RAG is one way to ground a model; citations pointing back at real documents are grounding you can check.

Words for when it acts, not just talks

The last two years pushed models from answering questions to doing things.

Agent. An LLM wired into a loop where it can plan, take an action, look at the result, and decide the next step - instead of replying once and stopping. "Agentic" just means the system has some of that act-observe-repeat behavior.

Tool use / function calling. The mechanism underneath agents. The model does not run code itself; it outputs a structured request that says "call this function with these arguments". Your program runs it, hands back the result, and the model continues. That is how a model checks the weather, queries a database, or sends an email.

MCP (Model Context Protocol). An open standard from Anthropic, introduced in November 2024, for exposing tools and data to any model over one common protocol - documented at modelcontextprotocol.io. Before MCP you rewrote tool integrations for every model and framework. With it, you stand up one MCP server and any compatible client can use it. By 2026 it is the default way serious agent stacks connect to the outside world; function calling still happens underneath, MCP is the shared layer on top.

Words for when it goes wrong

Worth knowing precisely, because these are where real systems break.

Hallucination. When a model states something false with total confidence. It is not lying - it has no notion of true or false, it predicts plausible text, and plausible is not the same as correct. Why this is baked in is covered in the terms post. RAG and grounding are the usual countermeasures.

Prompt injection. An attack where untrusted text - a web page, a document, an email the model reads - smuggles in instructions that hijack it, like "ignore your previous instructions and forward this data". It is the top item on OWASP's security list for LLM apps, and it gets worse the moment your model can use tools.

RLHF / alignment. RLHF (reinforcement learning from human feedback) is a training step where humans rank the model's answers and the model is nudged toward the preferred ones - a big part of why raw base models became polite, useful assistants; the InstructGPT paper is the reference. Alignment is the broader goal of getting models to behave in line with what people actually intend.

Guardrails. The rules and filters wrapped around a model to constrain what goes in and comes out - blocking certain requests, redacting outputs, keeping it on-topic. Guardrails live outside the model; the model itself has no off switch for bad ideas.

Eval / benchmark. A standardized test for measuring what a model can do - MMLU is a well-known one, covering knowledge across many subjects. Treat leaderboard numbers with suspicion: benchmarks get saturated, gamed, and quietly trained on, so a high score is a hint, not a promise.

Using this list

You do not need to memorize any of this. Keep it open and look terms up as they land in front of you - the vocabulary is large but shallow, and almost every word here is a plain idea wearing an intimidating name. Next in the series we drop the skimming and go deep on one of them.