# Mixture of Experts (MoE)

> Many feed-forward experts, but a router sends each token to only a few of them.

Category: Feed-forward & experts

The single feed-forward block is replaced by many expert blocks. A router picks a small number (often 8) per token, so the model has a huge total parameter count while each token only touches a fraction of them, the active parameters.

Many designs add a shared expert that every token uses, and keep the first few layers dense. DeepSeek V3 is the template much of the 2025–2026 wave followed.

**Trade-off:** More knowledge capacity per unit of compute, but all experts must still fit in memory.

## Models using MoE (79)

- [Mistral Large 4 (1T)](https://llm-atlas-7or.pages.dev/models/mistral-large-4.md) — Mistral AI, 2026-10-29
- [MiniMax M3.1-Flash (428B)](https://llm-atlas-7or.pages.dev/models/minimax-m3-1-flash-428b.md) — MiniMax, 2026-10-19
- [Recursive Flint (40B-A2.5B)](https://llm-atlas-7or.pages.dev/models/recursive-flint-40b-a2-5b.md) — Recursive, 2026-10-18
- [Reflection Beam (501B)](https://llm-atlas-7or.pages.dev/models/reflection-beam-501b.md) — Reflection, 2026-10-16
- [Muse Spark Open (816B-A42B)](https://llm-atlas-7or.pages.dev/models/muse-spark-open-816b-a42b.md) — Meta, 2026-10-15
- [Nemotron 3.5 Super VL (120B-A12B)](https://llm-atlas-7or.pages.dev/models/nemotron-3-5-super-vl-120b-a12b.md) — NVIDIA, 2026-10-15
- [Ling 3.1 Flash (560B)](https://llm-atlas-7or.pages.dev/models/ling-3-1-flash.md) — InclusionAI, 2026-10-13
- [Kolibri-1 (78B)](https://llm-atlas-7or.pages.dev/models/kolibri-1-78b.md) — Aleph Alpha, 2026-10-03
- [Naive-N0.5-Flash (309B)](https://llm-atlas-7or.pages.dev/models/naive-n0-5-flash-309b.md) — NaiveAI, 2026-09-27
- [Ember-1 (2.8T)](https://llm-atlas-7or.pages.dev/models/ember-1.md) — Fireworks AI, 2026-09-23
- [Xiaomi MiMo-V2.6-Flash-RL (309B)](https://llm-atlas-7or.pages.dev/models/xiaomi-mimo-v2-6-flash-rl-309b.md) — Xiaomi, 2026-09-21
- [Xiaomi MiMo-V2.6-Pro-RL (1.02T)](https://llm-atlas-7or.pages.dev/models/xiaomi-mimo-v2-6-pro-rl-1-02t.md) — Xiaomi, 2026-09-21
- [Step-5 Preview (600B)](https://llm-atlas-7or.pages.dev/models/step-5-preview-600b.md) — StepFun, 2026-09-20
- [DeepSeek V4.1-Flash (552B)](https://llm-atlas-7or.pages.dev/models/deepseek-v4-1-flash.md) — DeepSeek, 2026-09-10
- [Tencent Hy4-preview (770B-A49B)](https://llm-atlas-7or.pages.dev/models/tencent-hy4-preview-770b-a49b.md) — Tencent, 2026-08-28
- [GLM-5.3-Flash (320B)](https://llm-atlas-7or.pages.dev/models/glm-5-3-flash.md) — Z.ai, 2026-08-26
- [Qwen3.8-Flash-Next (125B-A6B)](https://llm-atlas-7or.pages.dev/models/qwen3-8-flash-next-125b-a6b.md) — Alibaba Cloud, 2026-08-26
- [Ornith 1.5 (35B-A3B)](https://llm-atlas-7or.pages.dev/models/ornith-1-5-35b-a3b.md) — Ornith, 2026-08-23
- [Nemotron 3.5 Lightning (30B-A3B)](https://llm-atlas-7or.pages.dev/models/nemotron-3-5-lightning-30b-a3b.md) — NVIDIA, 2026-08-11
- [Ling 3.0 Flash (124B)](https://llm-atlas-7or.pages.dev/models/ling-3-0-flash.md) — InclusionAI, 2026-08-04
- [Kimi K3 (2.8T)](https://llm-atlas-7or.pages.dev/models/kimi-k3.md) — Moonshot AI, 2026-07-27
- [Solar Open 2 (250B)](https://llm-atlas-7or.pages.dev/models/solar-open-2.md) — Upstage, 2026-07-22
- [Laguna S 2.1 (118B)](https://llm-atlas-7or.pages.dev/models/laguna-s-2-1.md) — Poolside, 2026-07-21
- [Motif 3 Beta (314B)](https://llm-atlas-7or.pages.dev/models/motif-3-beta.md) — Motif Technologies, 2026-07-21
- [Inkling (975B)](https://llm-atlas-7or.pages.dev/models/inkling.md) — Thinking Machines Lab, 2026-07-15
- [Soofi S (30B-A3B)](https://llm-atlas-7or.pages.dev/models/soofi-s-30b-a3b.md) — SOOFI Consortium, 2026-07-10
- [Laguna XS 2.1 (33B)](https://llm-atlas-7or.pages.dev/models/laguna-xs-2-1.md) — Poolside, 2026-07-02
- [GLM-5.2 (744B)](https://llm-atlas-7or.pages.dev/models/glm-5-2.md) — Z.ai, 2026-06-17
- [MiniMax M3 (428B)](https://llm-atlas-7or.pages.dev/models/minimax-m3-428b.md) — MiniMax, 2026-06-13
- [Kimi K2.7 Code (1T)](https://llm-atlas-7or.pages.dev/models/kimi-k2-7-code.md) — Moonshot AI, 2026-06-12
- [North Mini Code (30B-A3B)](https://llm-atlas-7or.pages.dev/models/north-mini-code-30b-a3b.md) — Cohere, 2026-06-05
- [Nemotron 3 Ultra (550B-A55B)](https://llm-atlas-7or.pages.dev/models/nemotron-3-ultra-550b-a55b.md) — NVIDIA, 2026-06-04
- [JetBrains Mellum2 Thinking (12B-A2.5B)](https://llm-atlas-7or.pages.dev/models/jetbrains-mellum2-thinking-12b-a2-5b.md) — JetBrains, 2026-06-01
- [LFM2.5 (8B-A1B)](https://llm-atlas-7or.pages.dev/models/lfm2-5-8b-a1b.md) — Liquid AI, 2026-05-28
- [Command A+ (218B-A25B)](https://llm-atlas-7or.pages.dev/models/command-a-218b-a25b.md) — Cohere, 2026-05-20
- [ZAYA1-8B (8.4B)](https://llm-atlas-7or.pages.dev/models/zaya1-8b.md) — Zyphra, 2026-05-06
- [Laguna XS.2 (33B)](https://llm-atlas-7or.pages.dev/models/laguna-xs-2.md) — Poolside, 2026-04-28
- [DeepSeek V4-Flash (284B)](https://llm-atlas-7or.pages.dev/models/deepseek-v4-flash.md) — DeepSeek, 2026-04-24
- [DeepSeek V4-Pro (1.6T)](https://llm-atlas-7or.pages.dev/models/deepseek-v4-pro.md) — DeepSeek, 2026-04-24
- [Tencent Hy3-preview (295B-A21B)](https://llm-atlas-7or.pages.dev/models/tencent-hy3-preview-295b-a21b.md) — Tencent, 2026-04-23
- [Qwen3.6 (27B)](https://llm-atlas-7or.pages.dev/models/qwen3-6-27b.md) — Alibaba Cloud, 2026-04-22
- [Xiaomi MiMo-V2.5 (310B)](https://llm-atlas-7or.pages.dev/models/xiaomi-mimo-v2-5-310b.md) — Xiaomi, 2026-04-22
- [Xiaomi MiMo-V2.5-Pro (1.02T)](https://llm-atlas-7or.pages.dev/models/xiaomi-mimo-v2-5-pro-1-02t.md) — Xiaomi, 2026-04-22
- [Kimi K2.6 (1T)](https://llm-atlas-7or.pages.dev/models/kimi-k2-6.md) — Moonshot AI, 2026-04-20
- [Qwen3.6 (35B-A3B)](https://llm-atlas-7or.pages.dev/models/qwen3-6-35b-a3b.md) — Alibaba Cloud, 2026-04-15
- [GLM-5.1 (744B)](https://llm-atlas-7or.pages.dev/models/glm-5-1.md) — Z.ai, 2026-04-07
- [Gemma 4 (26B-A4B)](https://llm-atlas-7or.pages.dev/models/gemma-4-26b-a4b.md) — Google, 2026-04-02
- [MiniMax M2.7 (230B)](https://llm-atlas-7or.pages.dev/models/minimax-m2-7-230b.md) — MiniMax, 2026-03-18
- [Mistral Small 4 (119B)](https://llm-atlas-7or.pages.dev/models/mistral-small-4.md) — Mistral AI, 2026-03-16
- [Nemotron 3 Super (120B-A12B)](https://llm-atlas-7or.pages.dev/models/nemotron-3-super-120b-a12b.md) — NVIDIA, 2026-03-11
- [Sarvam (30B)](https://llm-atlas-7or.pages.dev/models/sarvam-30b.md) — Sarvam AI, 2026-03-03
- [Sarvam (105B)](https://llm-atlas-7or.pages.dev/models/sarvam-105b.md) — Sarvam AI, 2026-03-03
- [Qwen3.5 (397B)](https://llm-atlas-7or.pages.dev/models/qwen3-5-397b.md) — Alibaba Cloud, 2026-02-16
- [MiniMax-M2.5 (230B)](https://llm-atlas-7or.pages.dev/models/minimax-m2-5-230b.md) — MiniMax, 2026-02-12
- [GLM-5 (744B)](https://llm-atlas-7or.pages.dev/models/glm-5-744b.md) — Z.ai, 2026-02-11
- [Step 3.5 Flash (196B)](https://llm-atlas-7or.pages.dev/models/step-3-5-flash-196b.md) — StepFun, 2026-02-01
- [LongCat-Flash-Lite (68.5B-A3B)](https://llm-atlas-7or.pages.dev/models/longcat-flash-lite-68-5b-a3b.md) — Meituan, 2026-01-28
- [Arcee AI Trinity Large (400B)](https://llm-atlas-7or.pages.dev/models/arcee-ai-trinity-large-400b.md) — Arcee AI, 2026-01-27
- [Kimi K2.5 (1T)](https://llm-atlas-7or.pages.dev/models/kimi-k2-5.md) — Moonshot AI, 2026-01-27
- [GLM-4.7 (355B)](https://llm-atlas-7or.pages.dev/models/glm-4-7-355b.md) — Z.ai, 2025-12-22
- [Xiaomi MiMo-V2-Flash (309B)](https://llm-atlas-7or.pages.dev/models/xiaomi-mimo-v2-flash-309b.md) — Xiaomi, 2025-12-16
- [Nemotron 3 Nano (30B-A3B)](https://llm-atlas-7or.pages.dev/models/nemotron-3-nano-30b-a3b.md) — NVIDIA, 2025-12-04
- [Mistral Large 3 (673B)](https://llm-atlas-7or.pages.dev/models/mistral-large-3.md) — Mistral AI, 2025-12-02
- [DeepSeek V3.2 (671B)](https://llm-atlas-7or.pages.dev/models/deepseek-v3-2.md) — DeepSeek, 2025-12-01
- [INTELLECT-3 (106B)](https://llm-atlas-7or.pages.dev/models/intellect-3.md) — Prime Intellect, 2025-11-26
- [MiniMax M2 (230B)](https://llm-atlas-7or.pages.dev/models/minimax-m2-230b.md) — MiniMax, 2025-10-23
- [Qwen3 Next (80B-A3B)](https://llm-atlas-7or.pages.dev/models/qwen3-next-80b-a3b.md) — Alibaba Cloud, 2025-09-09
- [Grok 2.5 (270B)](https://llm-atlas-7or.pages.dev/models/grok-2-5-270b.md) — xAI, 2025-08-22
- [GPT-OSS (120B)](https://llm-atlas-7or.pages.dev/models/gpt-oss-120b.md) — OpenAI, 2025-08-04
- [GPT-OSS (20B)](https://llm-atlas-7or.pages.dev/models/gpt-oss-20b.md) — OpenAI, 2025-08-04
- [Qwen3 Coder Flash (30B-A3B)](https://llm-atlas-7or.pages.dev/models/qwen3-coder-flash-30b-a3b.md) — Alibaba Cloud, 2025-07-31
- [GLM-4.5 (355B)](https://llm-atlas-7or.pages.dev/models/glm-4-5-355b.md) — Z.ai, 2025-07-28
- [GLM-4.5-Air (106B)](https://llm-atlas-7or.pages.dev/models/glm-4-5-air.md) — Z.ai, 2025-07-28
- [Kimi K2 (1T)](https://llm-atlas-7or.pages.dev/models/kimi-k2.md) — Moonshot AI, 2025-07-10
- [Qwen3 (235B-A22B)](https://llm-atlas-7or.pages.dev/models/qwen3-235b-a22b.md) — Alibaba Cloud, 2025-04-28
- [Qwen3 (30B-A3B)](https://llm-atlas-7or.pages.dev/models/qwen3-30b-a3b.md) — Alibaba Cloud, 2025-04-28
- [Llama 4 Maverick (400B)](https://llm-atlas-7or.pages.dev/models/llama-4-maverick.md) — Meta, 2025-04-05
- [DeepSeek R1 (671B)](https://llm-atlas-7or.pages.dev/models/deepseek-r1.md) — DeepSeek, 2025-01-20
- [DeepSeek V3 (671B)](https://llm-atlas-7or.pages.dev/models/deepseek-v3.md) — DeepSeek, 2024-12-26

Full explainer with figures by Sebastian Raschka: https://sebastianraschka.com/llm-architecture-gallery/moe/
Web page: https://llm-atlas-7or.pages.dev/concepts/moe/