# Learn: from tokens to agent infrastructure

A field guide in 18 topics and 489 terms. Each topic page has an interactive figure and a definition for every term.

## How a model works

Tokens, the transformer block and how text is generated.

- [1. LLM fundamentals](https://llm-atlas-7or.pages.dev/learn/fundamentals.md): From raw text to a probability for every possible next token. Terms: Tokens, Tokenization, Tokenizer, Token IDs, Vocabulary, Vocabulary size, Token embeddings, Embedding dimension, Hidden states, Final hidden state, Positional embeddings, Autoregressive generation, Forward pass, Model parameters, Model weights, Weight matrices, Matrix multiplication, LM head, Output projection, Logits, Softmax, Probability distribution, Next-token prediction, Next-token generation, Detokenization.
- [2. Transformer architecture](https://llm-atlas-7or.pages.dev/learn/transformer.md): The repeated block that almost every modern LLM is built from. Terms: Decoder-only Transformer, Transformer blocks, Self-attention, Attention layers, Query (Q), Key (K), Value (V), Attention masks, Causal attention, Multi-head attention, Rotary Positional Embeddings (RoPE), RMSNorm, Layer normalization, Feed-forward network (FFN), Multi-layer perceptron (MLP), Residual connections, Residual addition, Final normalization, Linear layers, Model architecture.
- [3. Decoding & sampling](https://llm-atlas-7or.pages.dev/learn/decoding.md): How a probability distribution becomes an actual word. Terms: Decoding, Sampling, Sampling settings, Temperature, Temperature scaling, Top-k sampling, Top-p / nucleus sampling, Greedy decoding, Argmax, Repetition penalty, Frequency penalty, Logit processing, Logit masking, Constrained decoding, Structured generation, JSON-constrained decoding, Token selection, Stopping conditions, End-of-sequence token.

## Serving at scale

Inference internals, batching, parallelism, MoE and the GPU underneath.

- [4. LLM inference internals](https://llm-atlas-7or.pages.dev/learn/inference.md): Why the first token is slow and the rest are fast. Terms: Inference, Prefill phase, Decode phase, Prompt processing, Time to first token (TTFT), Time per output token (TPOT), Inter-token latency, KV cache, KV-cache reuse, KV-cache allocation, KV-cache capacity, Prefix caching, Prompt caching, Cache eviction, Cache preemption, Attention computation, Compute-bound workloads, Memory-bandwidth-bound workloads, Speculative decoding, Draft model, Token acceptance, Model execution overhead.
- [5. Batching & scheduling](https://llm-atlas-7or.pages.dev/learn/batching.md): Sharing one GPU between many conversations without making anyone wait too long. Terms: Batching, Sequence batching, Static batching, Dynamic batching, Continuous batching, Batch size, Batched forward pass, Batch scheduling, Request admission, Inference scheduler, Request queue, Queueing delay, Concurrency, Concurrent inference, Active sequences, Sequence completion, Dynamic batch membership, Arithmetic intensity, Weight reuse, GPU utilization, Scheduling policy, Request preemption, Tail latency, Throughput-versus-latency trade-off.
- [6. Distributed LLM inference](https://llm-atlas-7or.pages.dev/learn/distributed.md): When one GPU is not enough: splitting, copying and routing models across a cluster. Terms: Distributed inference, Model parallelism, Tensor parallelism, Pipeline parallelism, Data parallelism, Data-parallel serving, Expert parallelism, Model sharding, Model replication, Multi-GPU inference, Multi-node inference, Distributed model serving, Distributed request routing, Cache-aware routing, Prefix-aware routing, Cluster scheduling, Global routing, Load balancing, Capacity-aware routing, Prefill/decode disaggregation, Disaggregated serving, Distributed KV caching, High-speed interconnects, Multi-tenant inference, Inference quality of service (QoS), Admission control, Autoscaling, High availability, Accelerator-aware scheduling.
- [7. Mixture of Experts (MoE)](https://llm-atlas-7or.pages.dev/learn/moe.md): A huge model that only wakes up a small part of itself for each token. Terms: Mixture of Experts (MoE), Dense models, Sparse MoE models, Experts, Expert MLPs, MoE routing, Router network, Routing logits, Gating network, Gating weights, Top-k expert selection, Top-2 routing, Active parameters, Total parameters, Expert dispatch, Expert combine, Token permutation, Grouped matrix multiplication, Expert load balancing, Expert placement, Inter-GPU communication, All-to-all communication, Routing imbalance, Sparse computation, Expert parallelism.
- [8. GPU & CUDA engineering](https://llm-atlas-7or.pages.dev/learn/gpu.md): The hardware realities that decide how fast a model can run. Terms: GPU computing, NVIDIA GPU, CUDA, CUDA C++, Triton language, GPU kernels, Kernel launches, Kernel fusion, Fused sampling, GPU memory, VRAM, GPU memory hierarchy, Memory bandwidth, Memory traffic, Memory locality, GPU compute utilization, Memory utilization, CPU–GPU data transfers, Tensor operations, Matrix-vector multiplication, Matrix-matrix multiplication, Parallel execution, GPU profiling, Kernel profiling, Accelerator clusters, NVIDIA H100, AWS Trainium, Google TPU, Hardware-specific optimization.

## Agents

The agent loop, tool calling and managing context and memory.

- [9. Agent architecture & internals](https://llm-atlas-7or.pages.dev/learn/agents.md): A model in a loop, with tools, state and a reason to stop. Terms: AI agents, Agentic systems, Agent systems engineering, Agent architecture, Agent runtime, Agent harness, Agent control loop, Reasoning loop, Agent orchestration, Tool-calling loop, Agent state machines, Agent lifecycle, Agent sessions, Agent identity, Agent configuration, Agent instructions, Agent capabilities, Agent state, Tool selection, Tool invocation, Tool result handling, Tool dispatch, Agent termination, Subagents, Agent delegation, Multi-agent systems, Multi-agent collaboration, Agent coordination, Agent-to-agent communication, Task decomposition, Autonomous task execution.
- [10. Tool calling & protocols](https://llm-atlas-7or.pages.dev/learn/tool-calling.md): How a model asks for something to be done, and how the answer gets back. Terms: Function calling, Tool definitions, Tool schemas, Tool descriptions, Function arguments, Tool-call tokens, Special tokens, Control tokens, Message boundary tokens, Tool routing tokens, Chat templates, Structured model outputs, JSON Schema, JSON parsing, Argument validation, Tool authorization, Tool permissions, Tool execution, Tool-call errors, Hallucinated tools, OpenAI-compatible API, AG-UI protocol, Model Context Protocol (MCP), Agent-to-application communication, Agent-to-tool communication, Tool-call finish reasons, Forced tool selection, Tool choice.
- [11. Context, memory & persistence](https://llm-atlas-7or.pages.dev/learn/context-memory.md): What the model can see right now, and what the system remembers for later. Terms: Context window, Context length, Context management, Context assembly, Short-term memory, Long-term memory, Persistent memory, Conversation history, Message history, Memory retrieval, Retrieval systems, Context compaction, Context summarization, Context truncation, Token budgets, Persistent agent state, Execution history, Checkpointing, State snapshots, State recovery, Execution replay, Deterministic replay, Nondeterministic execution, Durable state, Persistent workflows.

## Production systems

Backends, security, sandboxing and observability.

- [12. Backend & distributed systems](https://llm-atlas-7or.pages.dev/learn/backend.md): The plumbing that keeps thousands of agent runs moving and correct. Terms: Backend engineering, Control plane, Execution plane, Inference plane, API gateway, Model gateway, Service boundaries, Modular monolith, Microservices, Distributed systems, Distributed orchestration, Worker processes, Worker pools, Task scheduling, Task queues, Message queues, Message brokers, Asynchronous messaging, Event-driven architecture, Event messaging, Distributed coordination, Concurrency limits, Work distribution, Distributed locks, Leases, Lease-based ownership, Work stealing, Backpressure, Idempotency, Idempotency keys, Transactional outbox, Compensating actions, Exactly-once execution, At-least-once execution, Fault tolerance, Distributed recovery, Resource allocation.
- [13. Reliability, security & sandboxing](https://llm-atlas-7or.pages.dev/learn/security.md): Letting an agent act in the world without letting it break things. Terms: Tool isolation, Execution sandbox, Sandboxed execution, Docker containers, MicroVMs, Process isolation, Workspace isolation, Filesystem boundaries, Browser isolation, Security policies, Capability security, Authorization, Authentication, Permission enforcement, Policy gateway, Fail-closed security, Audit logging, Audit trails, Human approval, Human-in-the-loop, Secrets management, Credential encryption, Error handling, Retries, Retry amplification, Timeouts, Cancellation, Crash recovery, Failure handling, Replay safety, External side effects, Rate limiting, Resource limits.
- [14. Observability & performance](https://llm-atlas-7or.pages.dev/learn/observability.md): Seeing where the time, tokens and money actually go. Terms: Observability, Distributed tracing, Execution traces, Trace correlation, Structured logging, Metrics, GPU telemetry, Performance profiling, Inference profiling, Latency, End-to-end latency, p95 latency, Tail latency, Throughput, Tokens per second, Requests per second, Task completion rate, Time to first token, Scheduling delay, Queue depth, GPU memory usage, GPU utilization, KV-cache utilization, Cost tracking, Token usage, Load testing, Benchmarking, Performance regressions, Bottleneck analysis, Capacity planning, Agent evaluations, Behavioral correctness, Regression testing.

## Training, tools & research

How models are trained and edited, the tool stack, and open problems.

- [15. Model training & post-training](https://llm-atlas-7or.pages.dev/learn/training.md): How a pile of text becomes a model that follows instructions and uses tools. Terms: Pretraining, Supervised fine-tuning (SFT), Post-training, Tool-use training, Training examples, Learning from demonstrations, Gradient descent, Backpropagation, Loss functions, Optimizers, Adam optimizer, Training datasets, Model behavior, Model capabilities, Training feedback, Model evaluation, Benchmark evaluation.
- [16. Weight-space programming & interpretability](https://llm-atlas-7or.pages.dev/learn/interpretability.md): Reading and editing what a network has learned, directly in its weights. Terms: Weight-space programming, Weight-space reading, Weight-space writing, Weight-space editing, Direct weight modification, Neural network parameter inspection, Numerical weight edits, Weight matrices, Tensor inspection, Matrix properties, Activation patterns, Singular vectors, Singular value decomposition (SVD), Behavioral directions, MLP weight editing, Mechanistic interpretability, Model editing, Targeted model intervention, Parameter-efficient modifications, Failure-mode localization, Repetitive reasoning loops, Automated neural network engineering, MATH-500 benchmark, Model regression evaluation.
- [17. Infrastructure & development tools](https://llm-atlas-7or.pages.dev/learn/stack.md): The tools a modern agent and inference stack is actually built with. Terms: ASP.NET Core, C# / .NET, Entity Framework Core, PostgreSQL, JSONB, Redis, NATS JetStream, Temporal, Docker, Docker Compose, Kubernetes, Terraform, GitHub Actions, OpenTelemetry, Prometheus, Grafana, Grafana Tempo, Grafana Loki, NVIDIA DCGM Exporter, xUnit, Testcontainers, k6, Next.js, TypeScript, shadcn/ui, Python, vLLM, SGLang, llama.cpp, TensorRT-LLM, llm-d, Microsoft Agent Framework, LangGraph, CrewAI, Pydantic AI, Google ADK, Mastra, CopilotKit, OpenClaw, Hermes Agent, Megatron-LM, Qwen, Llama, DeepSeek, CUDA, Triton.
- [18. Advanced systems research questions](https://llm-atlas-7or.pages.dev/learn/research.md): Open problems where agents and inference infrastructure meet. Terms: Inference-aware agent scheduling, Agent-aware inference serving, Joint agent/inference scheduling, GPU-aware orchestration, Multi-tenant agent infrastructure, Bursty agent workloads, Agent workload characterization, Agent concurrency optimization, Task-throughput optimization, Cross-layer optimization, Scheduling fairness, Cost-aware model routing, Model placement, Accelerator placement, Resource-aware scheduling, Context-aware routing, Agent execution profiling, Dynamic admission control, Compute resource contention, Inference cost optimization, Distributed agent benchmarking, Agent-runtime operating-system analogy.

All definitions as JSON: https://llm-atlas-7or.pages.dev/api/glossary.json