18 topics · 489 terms
From tokens to agent infrastructure
01How a model works
Tokens, the transformer block and how text is generated.
- 1 · 25 terms
LLM fundamentals
From raw text to a probability for every possible next token.
Figure: Text in, next token out
- 2 · 20 terms
Transformer architecture
The repeated block that almost every modern LLM is built from.
Figure: Who attends to whom
- 3 · 19 terms
Decoding & sampling
How a probability distribution becomes an actual word.
Figure: Sampling playground
02Serving at scale
Inference internals, batching, parallelism, MoE and the GPU underneath.
- 4 · 22 terms
LLM inference internals
Why the first token is slow and the rest are fast.
Figure: Prefill, then decode
- 5 · 24 terms
Batching & scheduling
Sharing one GPU between many conversations without making anyone wait too long.
Figure: Static vs continuous batching
- 6 · 29 terms
Distributed LLM inference
When one GPU is not enough: splitting, copying and routing models across a cluster.
Figure: Four ways to split a model
- 7 · 25 terms
Mixture of Experts (MoE)
A huge model that only wakes up a small part of itself for each token.
Figure: A router at work
- 8 · 29 terms
GPU & CUDA engineering
The hardware realities that decide how fast a model can run.
Figure: The roofline
03Agents
The agent loop, tool calling and managing context and memory.
- 9 · 31 terms
Agent architecture & internals
A model in a loop, with tools, state and a reason to stop.
Figure: The agent loop
- 10 · 28 terms
Tool calling & protocols
How a model asks for something to be done, and how the answer gets back.
Figure: Anatomy of a tool call
- 11 · 25 terms
Context, memory & persistence
What the model can see right now, and what the system remembers for later.
Figure: Filling a context window
04Production systems
Backends, security, sandboxing and observability.
- 12 · 37 terms
Backend & distributed systems
The plumbing that keeps thousands of agent runs moving and correct.
Figure: Three planes of an agent platform
- 13 · 33 terms
Reliability, security & sandboxing
Letting an agent act in the world without letting it break things.
Figure: Defence in layers
- 14 · 33 terms
Observability & performance
Seeing where the time, tokens and money actually go.
Figure: One agent run, traced
05Training, tools & research
How models are trained and edited, the tool stack, and open problems.
- 15 · 17 terms
Model training & post-training
How a pile of text becomes a model that follows instructions and uses tools.
Figure: Gradient descent, by hand
- 16 · 24 terms
Weight-space programming & interpretability
Reading and editing what a network has learned, directly in its weights.
Figure: A matrix, one direction at a time
- 17 · 46 terms
Infrastructure & development tools
The tools a modern agent and inference stack is actually built with.
Figure: The stack, layer by layer
- 18 · 22 terms
Advanced systems research questions
Open problems where agents and inference infrastructure meet.
Figure: The agent runtime as an operating system
Reading this as an agent?
Every topic is also available as Markdown, and all 489 definitions are in one JSON file.