# Manifold-constrained Hyper-Connections (mHC)

> Several parallel residual streams with learned, constrained mixing between them.

Category: Residual stream

The single residual stream is widened into several streams (four in DeepSeek V4). Learned mappings decide how sublayer inputs are read from, and outputs written back to, these streams.

Unconstrained hyper-connections can amplify signals and destabilise training; mHC constrains the mixing matrix to a well-behaved manifold so information is mixed without blowing up. DeepSeek V4.1 introduced a cheaper single-pass variant.

**Trade-off:** A richer residual path for a modest training overhead.

## Models using mHC (6)

- [DeepSeek V4.1-Flash (552B)](https://llm-atlas-7or.pages.dev/models/deepseek-v4-1-flash.md) — DeepSeek, 2026-09-10
- [Tencent Hy4-preview (770B-A49B)](https://llm-atlas-7or.pages.dev/models/tencent-hy4-preview-770b-a49b.md) — Tencent, 2026-08-28
- [GLM-5.3-Flash (320B)](https://llm-atlas-7or.pages.dev/models/glm-5-3-flash.md) — Z.ai, 2026-08-26
- [Motif 3 Beta (314B)](https://llm-atlas-7or.pages.dev/models/motif-3-beta.md) — Motif Technologies, 2026-07-21
- [DeepSeek V4-Flash (284B)](https://llm-atlas-7or.pages.dev/models/deepseek-v4-flash.md) — DeepSeek, 2026-04-24
- [DeepSeek V4-Pro (1.6T)](https://llm-atlas-7or.pages.dev/models/deepseek-v4-pro.md) — DeepSeek, 2026-04-24

Full explainer with figures by Sebastian Raschka: https://sebastianraschka.com/llm-architecture-gallery/mhc/
Web page: https://llm-atlas-7or.pages.dev/concepts/mhc/