# Multi-Token Prediction (MTP)

> Extra heads learn to predict several future tokens, and can draft tokens at inference.

Category: Training objectives

Besides the next token, small additional modules predict tokens further ahead. DeepSeek's version keeps a causal chain where each module conditions on the previous prediction.

The extra targets give a denser training signal, and at inference the MTP modules act as a built-in draft model for speculative decoding.

**Trade-off:** Denser training signal and faster decoding, for some extra parameters.

## Models using MTP (34)

- [Nemotron 3.5 Super VL (120B-A12B)](https://llm-atlas-7or.pages.dev/models/nemotron-3-5-super-vl-120b-a12b.md) — NVIDIA, 2026-10-15
- [Ling 3.1 Flash (560B)](https://llm-atlas-7or.pages.dev/models/ling-3-1-flash.md) — InclusionAI, 2026-10-13
- [Xiaomi MiMo-V2.6-Flash-RL (309B)](https://llm-atlas-7or.pages.dev/models/xiaomi-mimo-v2-6-flash-rl-309b.md) — Xiaomi, 2026-09-21
- [Xiaomi MiMo-V2.6-Pro-RL (1.02T)](https://llm-atlas-7or.pages.dev/models/xiaomi-mimo-v2-6-pro-rl-1-02t.md) — Xiaomi, 2026-09-21
- [Step-5 Preview (600B)](https://llm-atlas-7or.pages.dev/models/step-5-preview-600b.md) — StepFun, 2026-09-20
- [Tencent Hy4-preview (770B-A49B)](https://llm-atlas-7or.pages.dev/models/tencent-hy4-preview-770b-a49b.md) — Tencent, 2026-08-28
- [GLM-5.3-Flash (320B)](https://llm-atlas-7or.pages.dev/models/glm-5-3-flash.md) — Z.ai, 2026-08-26
- [Qwen3.8-Flash-Next (125B-A6B)](https://llm-atlas-7or.pages.dev/models/qwen3-8-flash-next-125b-a6b.md) — Alibaba Cloud, 2026-08-26
- [Ornith 1.5 (35B-A3B)](https://llm-atlas-7or.pages.dev/models/ornith-1-5-35b-a3b.md) — Ornith, 2026-08-23
- [Qwen3.8 (27B)](https://llm-atlas-7or.pages.dev/models/qwen3-8-27b.md) — Alibaba Cloud, 2026-08-14
- [Nemotron 3.5 Lightning (30B-A3B)](https://llm-atlas-7or.pages.dev/models/nemotron-3-5-lightning-30b-a3b.md) — NVIDIA, 2026-08-11
- [Ling 3.0 Flash (124B)](https://llm-atlas-7or.pages.dev/models/ling-3-0-flash.md) — InclusionAI, 2026-08-04
- [Motif 3 Beta (314B)](https://llm-atlas-7or.pages.dev/models/motif-3-beta.md) — Motif Technologies, 2026-07-21
- [Inkling (975B)](https://llm-atlas-7or.pages.dev/models/inkling.md) — Thinking Machines Lab, 2026-07-15
- [DeepSeek V4-Flash (284B)](https://llm-atlas-7or.pages.dev/models/deepseek-v4-flash.md) — DeepSeek, 2026-04-24
- [DeepSeek V4-Pro (1.6T)](https://llm-atlas-7or.pages.dev/models/deepseek-v4-pro.md) — DeepSeek, 2026-04-24
- [Ling 2.6 (1T)](https://llm-atlas-7or.pages.dev/models/ling-2-6-1t.md) — InclusionAI, 2026-04-23
- [Tencent Hy3-preview (295B-A21B)](https://llm-atlas-7or.pages.dev/models/tencent-hy3-preview-295b-a21b.md) — Tencent, 2026-04-23
- [Xiaomi MiMo-V2.5 (310B)](https://llm-atlas-7or.pages.dev/models/xiaomi-mimo-v2-5-310b.md) — Xiaomi, 2026-04-22
- [GLM-5.1 (744B)](https://llm-atlas-7or.pages.dev/models/glm-5-1.md) — Z.ai, 2026-04-07
- [MiniMax M2.7 (230B)](https://llm-atlas-7or.pages.dev/models/minimax-m2-7-230b.md) — MiniMax, 2026-03-18
- [Nemotron 3 Super (120B-A12B)](https://llm-atlas-7or.pages.dev/models/nemotron-3-super-120b-a12b.md) — NVIDIA, 2026-03-11
- [MiniMax-M2.5 (230B)](https://llm-atlas-7or.pages.dev/models/minimax-m2-5-230b.md) — MiniMax, 2026-02-12
- [GLM-5 (744B)](https://llm-atlas-7or.pages.dev/models/glm-5-744b.md) — Z.ai, 2026-02-11
- [Step 3.5 Flash (196B)](https://llm-atlas-7or.pages.dev/models/step-3-5-flash-196b.md) — StepFun, 2026-02-01
- [GLM-4.7 (355B)](https://llm-atlas-7or.pages.dev/models/glm-4-7-355b.md) — Z.ai, 2025-12-22
- [Xiaomi MiMo-V2-Flash (309B)](https://llm-atlas-7or.pages.dev/models/xiaomi-mimo-v2-flash-309b.md) — Xiaomi, 2025-12-16
- [DeepSeek V3.2 (671B)](https://llm-atlas-7or.pages.dev/models/deepseek-v3-2.md) — DeepSeek, 2025-12-01
- [MiniMax M2 (230B)](https://llm-atlas-7or.pages.dev/models/minimax-m2-230b.md) — MiniMax, 2025-10-23
- [Qwen3 Next (80B-A3B)](https://llm-atlas-7or.pages.dev/models/qwen3-next-80b-a3b.md) — Alibaba Cloud, 2025-09-09
- [GLM-4.5 (355B)](https://llm-atlas-7or.pages.dev/models/glm-4-5-355b.md) — Z.ai, 2025-07-28
- [GLM-4.5-Air (106B)](https://llm-atlas-7or.pages.dev/models/glm-4-5-air.md) — Z.ai, 2025-07-28
- [DeepSeek R1 (671B)](https://llm-atlas-7or.pages.dev/models/deepseek-r1.md) — DeepSeek, 2025-01-20
- [DeepSeek V3 (671B)](https://llm-atlas-7or.pages.dev/models/deepseek-v3.md) — DeepSeek, 2024-12-26

Full explainer with figures by Sebastian Raschka: https://sebastianraschka.com/llm-architecture-gallery/mtp/
Web page: https://llm-atlas-7or.pages.dev/concepts/mtp/