# 11. Context, memory & persistence

> What the model can see right now, and what the system remembers for later.

Track: Agents. 25 terms.

A model has no memory between calls. Everything it knows about the task has to be in the context window on every request: instructions, tool definitions, the conversation so far, tool results and anything retrieved from storage. The window is large but finite, and filling it costs money and latency.

Context management is the art of choosing what goes in. Persistence is the other half: saving the agent's state durably, so a long task can survive a crash, be replayed for debugging or pick up again next week.

**Interactive figure on the web page: Filling a context window.** A 128K-token window split between instructions, tools, retrieved knowledge, conversation history and room for the reply. Add turns and the history grows until it no longer fits. Choose a strategy to see what happens: truncation drops the oldest turns, compaction folds them into a short summary.

## Terms

### Context window

The maximum number of tokens a model can attend to at once, prompt and output together, such as 128K or 1M tokens.

See: [related](https://llm-atlas-7or.pages.dev/models/?sort=context)

### Context length

The number of tokens actually in a given request. Cost and latency grow with it, and quality can slip well before the window is full.

### Context management

Deciding what goes into the limited context on each model call: which instructions, history, tool results and retrieved knowledge.

### Context assembly

Building the final prompt from its parts in a stable order: instructions, tool definitions, memory, retrieved documents and recent messages. A stable order also keeps prefix caching effective.

See: [related](https://llm-atlas-7or.pages.dev/learn/inference/#prefix-caching)

### Short-term memory

Information kept within the current context or session, mainly the recent conversation and working notes.

### Long-term memory

Information kept across sessions in external storage, such as files, databases or vector stores, and brought back when relevant.

### Persistent memory

Memory that survives restarts and sessions, such as user preferences or project facts written to durable storage.

### Conversation history

The sequence of past user, assistant and tool messages in a session.

### Message history

The raw list of messages sent to the model on each turn. Because the model is stateless, the whole history is sent again on every call, ideally with caching.

### Memory retrieval

Selecting the stored memories relevant to the current step, by recency, keywords or embedding similarity.

### Retrieval systems

Search infrastructure, such as BM25, vector search, hybrid search and rerankers, that finds documents to place into context, as in retrieval-augmented generation (RAG).

### Context compaction

Shrinking the context while keeping what matters: summarising old turns, dropping stale tool output, holding on to key decisions.

### Context summarization

Replacing a span of history with a model-written summary. It saves tokens but can lose details.

### Context truncation

Dropping the oldest or least important content when the context exceeds its budget. Simple, but information disappears abruptly.

### Token budgets

Allocations of the window to each part (instructions, tools, history, retrieval, output), enforced when the context is assembled.

### Persistent agent state

Agent state saved to durable storage so a run can pause, survive crashes and resume.

### Execution history

A record of every step an agent took: model calls, tool calls, results and decisions.

### Checkpointing

Saving the agent's state at defined points so it can resume from there instead of starting over.

### State snapshots

Point-in-time copies of the full state, useful for debugging, branching and rolling back.

### State recovery

Restoring an agent from its last checkpoint after a failure.

### Execution replay

Re-running a recorded execution, reusing the recorded model and tool results, to debug it or to recover.

### Deterministic replay

Replay that follows the same path every time because nondeterministic results, such as model outputs, clock reads and tool responses, are read from the log instead of recomputed. Workflow engines like Temporal depend on it.

### Nondeterministic execution

Steps that can give a different result each time: sampled model output, network calls, the current time. They must be recorded to make replay safe.

### Durable state

State stored where it survives process and machine failures, such as a database or an event log.

### Persistent workflows

Long-running workflows whose progress is recorded durably, so they can run for hours or days and survive restarts.

Web page: https://llm-atlas-7or.pages.dev/learn/context-memory/

_Definitions written for LLM Atlas._