# Gated Attention (Gated attention)

> A learned sigmoid gate scales the attention output before it re-enters the residual stream.

Category: Attention

After the softmax attention is computed, a gate derived from the layer input multiplies the result element-wise. The gate can damp heads or channels that are not useful for the current token.

Qwen3-Next and Qwen3.5 use it in their full-attention layers next to linear-attention layers; Trinity Large uses it in an attention-only stack. Reported benefits include better training stability and fewer attention-sink artefacts.

**Trade-off:** A cheap extra projection that adds control and stability to the attention output.

## Models using Gated attention (21)

- [LFM3 (1.7B)](https://llm-atlas-7or.pages.dev/models/lfm3-1-7b.md) — Liquid AI, 2026-10-27
- [Reflection Beam (501B)](https://llm-atlas-7or.pages.dev/models/reflection-beam-501b.md) — Reflection, 2026-10-16
- [Dory1.0 (4B)](https://llm-atlas-7or.pages.dev/models/dory1-0-4b.md) — NVIDIA, 2026-10-16
- [Muse Spark Open (816B-A42B)](https://llm-atlas-7or.pages.dev/models/muse-spark-open-816b-a42b.md) — Meta, 2026-10-15
- [Ling 3.1 Flash (560B)](https://llm-atlas-7or.pages.dev/models/ling-3-1-flash.md) — InclusionAI, 2026-10-13
- [Ember-1 (2.8T)](https://llm-atlas-7or.pages.dev/models/ember-1.md) — Fireworks AI, 2026-09-23
- [Step-5 Preview (600B)](https://llm-atlas-7or.pages.dev/models/step-5-preview-600b.md) — StepFun, 2026-09-20
- [Tencent Hy4-preview (770B-A49B)](https://llm-atlas-7or.pages.dev/models/tencent-hy4-preview-770b-a49b.md) — Tencent, 2026-08-28
- [Qwen3.8-Flash-Next (125B-A6B)](https://llm-atlas-7or.pages.dev/models/qwen3-8-flash-next-125b-a6b.md) — Alibaba Cloud, 2026-08-26
- [Ornith 1.5 (35B-A3B)](https://llm-atlas-7or.pages.dev/models/ornith-1-5-35b-a3b.md) — Ornith, 2026-08-23
- [Qwen3.8 (27B)](https://llm-atlas-7or.pages.dev/models/qwen3-8-27b.md) — Alibaba Cloud, 2026-08-14
- [Muse Glimmer (30B)](https://llm-atlas-7or.pages.dev/models/muse-glimmer-30b.md) — Meta, 2026-08-10
- [Ling 3.0 Flash (124B)](https://llm-atlas-7or.pages.dev/models/ling-3-0-flash.md) — InclusionAI, 2026-08-04
- [Kimi K3 (2.8T)](https://llm-atlas-7or.pages.dev/models/kimi-k3.md) — Moonshot AI, 2026-07-27
- [Solar Open 2 (250B)](https://llm-atlas-7or.pages.dev/models/solar-open-2.md) — Upstage, 2026-07-22
- [BTL-3 (27B)](https://llm-atlas-7or.pages.dev/models/btl-3-27b.md) — Bad Theory Labs, 2026-07-20
- [Qwen3.6 (27B)](https://llm-atlas-7or.pages.dev/models/qwen3-6-27b.md) — Alibaba Cloud, 2026-04-22
- [Qwen3.6 (35B-A3B)](https://llm-atlas-7or.pages.dev/models/qwen3-6-35b-a3b.md) — Alibaba Cloud, 2026-04-15
- [Qwen3.5 (397B)](https://llm-atlas-7or.pages.dev/models/qwen3-5-397b.md) — Alibaba Cloud, 2026-02-16
- [Arcee AI Trinity Large (400B)](https://llm-atlas-7or.pages.dev/models/arcee-ai-trinity-large-400b.md) — Arcee AI, 2026-01-27
- [Qwen3 Next (80B-A3B)](https://llm-atlas-7or.pages.dev/models/qwen3-next-80b-a3b.md) — Alibaba Cloud, 2025-09-09

Full explainer with figures by Sebastian Raschka: https://sebastianraschka.com/llm-architecture-gallery/gated-attention/
Web page: https://llm-atlas-7or.pages.dev/concepts/gated-attention/