# Hybrid Attention (Hybrid attention)

> Mixes cheap linear-attention or state-space layers with a few full-attention layers.

Category: Attention

Linear attention variants (Gated DeltaNet, Kimi Delta Attention, Lightning Attention) and state-space layers such as Mamba-2 keep a fixed-size recurrent state instead of a per-token KV cache, so their memory does not grow with context.

Because they are weaker at precise retrieval, hybrids interleave them with a smaller number of full-attention layers, for example three linear layers per full-attention layer in the Qwen pattern.

**Trade-off:** Much lower long-context memory and compute, trading away some exact recall in the linear layers.

## Models using Hybrid attention (24)

- [Nemotron 3.5 Super VL (120B-A12B)](https://llm-atlas-7or.pages.dev/models/nemotron-3-5-super-vl-120b-a12b.md) — NVIDIA, 2026-10-15
- [Ling 3.1 Flash (560B)](https://llm-atlas-7or.pages.dev/models/ling-3-1-flash.md) — InclusionAI, 2026-10-13
- [Ember-1 (2.8T)](https://llm-atlas-7or.pages.dev/models/ember-1.md) — Fireworks AI, 2026-09-23
- [GLM-5.3-Flash (320B)](https://llm-atlas-7or.pages.dev/models/glm-5-3-flash.md) — Z.ai, 2026-08-26
- [Qwen3.8-Flash-Next (125B-A6B)](https://llm-atlas-7or.pages.dev/models/qwen3-8-flash-next-125b-a6b.md) — Alibaba Cloud, 2026-08-26
- [Ornith 1.5 (35B-A3B)](https://llm-atlas-7or.pages.dev/models/ornith-1-5-35b-a3b.md) — Ornith, 2026-08-23
- [Qwen3.8 (27B)](https://llm-atlas-7or.pages.dev/models/qwen3-8-27b.md) — Alibaba Cloud, 2026-08-14
- [Nemotron 3.5 Lightning (30B-A3B)](https://llm-atlas-7or.pages.dev/models/nemotron-3-5-lightning-30b-a3b.md) — NVIDIA, 2026-08-11
- [Ling 3.0 Flash (124B)](https://llm-atlas-7or.pages.dev/models/ling-3-0-flash.md) — InclusionAI, 2026-08-04
- [Kimi K3 (2.8T)](https://llm-atlas-7or.pages.dev/models/kimi-k3.md) — Moonshot AI, 2026-07-27
- [Solar Open 2 (250B)](https://llm-atlas-7or.pages.dev/models/solar-open-2.md) — Upstage, 2026-07-22
- [BTL-3 (27B)](https://llm-atlas-7or.pages.dev/models/btl-3-27b.md) — Bad Theory Labs, 2026-07-20
- [Soofi S (30B-A3B)](https://llm-atlas-7or.pages.dev/models/soofi-s-30b-a3b.md) — SOOFI Consortium, 2026-07-10
- [Nemotron 3 Ultra (550B-A55B)](https://llm-atlas-7or.pages.dev/models/nemotron-3-ultra-550b-a55b.md) — NVIDIA, 2026-06-04
- [Ling 2.6 (1T)](https://llm-atlas-7or.pages.dev/models/ling-2-6-1t.md) — InclusionAI, 2026-04-23
- [Qwen3.6 (27B)](https://llm-atlas-7or.pages.dev/models/qwen3-6-27b.md) — Alibaba Cloud, 2026-04-22
- [Qwen3.6 (35B-A3B)](https://llm-atlas-7or.pages.dev/models/qwen3-6-35b-a3b.md) — Alibaba Cloud, 2026-04-15
- [Nemotron 3 Nano (4B)](https://llm-atlas-7or.pages.dev/models/nemotron-3-nano-4b.md) — NVIDIA, 2026-03-16
- [Nemotron 3 Super (120B-A12B)](https://llm-atlas-7or.pages.dev/models/nemotron-3-super-120b-a12b.md) — NVIDIA, 2026-03-11
- [Qwen3.5 (397B)](https://llm-atlas-7or.pages.dev/models/qwen3-5-397b.md) — Alibaba Cloud, 2026-02-16
- [Ling 2.5 (1T)](https://llm-atlas-7or.pages.dev/models/ling-2-5-1t.md) — InclusionAI, 2026-02-15
- [Nemotron 3 Nano (30B-A3B)](https://llm-atlas-7or.pages.dev/models/nemotron-3-nano-30b-a3b.md) — NVIDIA, 2025-12-04
- [Kimi Linear (48B-A3B)](https://llm-atlas-7or.pages.dev/models/kimi-linear-48b-a3b.md) — Moonshot AI, 2025-10-30
- [Qwen3 Next (80B-A3B)](https://llm-atlas-7or.pages.dev/models/qwen3-next-80b-a3b.md) — Alibaba Cloud, 2025-09-09

Full explainer with figures by Sebastian Raschka: https://sebastianraschka.com/llm-architecture-gallery/hybrid-attention/
Web page: https://llm-atlas-7or.pages.dev/concepts/hybrid-attention/