# Layer-wise attention budgeting (Attention budgeting)

> Different layers get different attention budgets: head counts, windows or full context.

Category: Attention

Rather than giving every layer an identical attention setup, the model spends more attention capacity in some layers and less in others.

Laguna XS.2 is the reference example: most of its 40 layers use a 512-token sliding window, while a minority keep full attention, and the head counts differ between the two kinds of layer.

**Trade-off:** Puts expensive global attention only where it earns its keep, cutting KV cache and compute.

## Models using Attention budgeting (3)

- [Laguna S 2.1 (118B)](https://llm-atlas-7or.pages.dev/models/laguna-s-2-1.md) — Poolside, 2026-07-21
- [Laguna XS 2.1 (33B)](https://llm-atlas-7or.pages.dev/models/laguna-xs-2-1.md) — Poolside, 2026-07-02
- [Laguna XS.2 (33B)](https://llm-atlas-7or.pages.dev/models/laguna-xs-2.md) — Poolside, 2026-04-28

Full explainer with figures by Sebastian Raschka: https://sebastianraschka.com/llm-architecture-gallery/attention-budgeting/
Web page: https://llm-atlas-7or.pages.dev/concepts/attention-budgeting/