Thesis
MTP (Multi-Token Prediction) fundamentally alters the compute-to-memory ratio of inference workloads, shifting them from memory-bound toward compute-bound. This is not a marginal optimization — it restructures the ROI calculus across chips, interconnects, and memory hierarchies.
DeepSeek remains a high-end player with conviction: committed to cost-efficiency, targeting the most expensive bottleneck, and pursuing technology that democratizes access.
1. Compute Intensity: Inference Approaches Training
Traditional autoregressive decode is the canonical memory-bound workload: minimal FLOPs per token, massive KV-Cache traffic. MTP inverts this by amortizing the same KV-Cache across multiple predicted tokens.
| Metric | Standard Decode | MTP Decode (k steps) |
|---|---|---|
| Compute per token | 1× FLOPs | ≈k× FLOPs |
| KV-Cache access | 1× | 1× (shared) |
| Arithmetic intensity | Low | Linear growth in k |
Key insight: As MTP depth $k$ grows, inference arithmetic intensity approaches — and can theoretically exceed — training intensity, because MTP's multi-step prediction path generates more FLOPs per byte of HBM traffic than single-step training.
The acceptance rate caveat: The $k\times$ compute growth is a theoretical upper bound. Realized gains depend on acceptance rate — the fraction of predicted tokens that pass verification. DSpark (DeepSeek + PKU, 2026) empirically shows that naive deep drafters (essentially stacked MTP) suffer from rapid acceptance decay ("suffix decay"): deeper predictions have progressively lower acceptance rates [[5]](#ref-5). This is precisely why DeepSeek abandoned the "deeper MTP" route in favor of semi-autoregressive drafting with confidence-scheduled verification. In practice, production MTP deployments (DeepSeek-V3) use only 1-2 auxiliary depths with a 0.1 loss scaling factor [[1]](#ref-1) — the "large MTP (k=5+)" scenario remains theoretical, not production-validated.
Industry context: NVIDIA's GPU roadmap shows memory bandwidth growing slower than compute across generations — the structural gap MTP exploits:
⚠️ 规格待 review — 下表数据基于官方发布信息整理，建议读者点击 Source 链接人工核实关键数字。
| GPU | Architecture | Memory | BW | FP8 Dense | BW/Compute Ratio | Source |
|---|---|---|---|---|---|---|
| H100 SXM | Hopper | 80 GB HBM3 | 3.35 TB/s | 989 TFLOPS | 3.4 GB/s per TFLOP | NVIDIA H100 |
| H200 SXM | Hopper | 141 GB HBM3e | 4.8 TB/s | 989 TFLOPS | 4.9 GB/s per TFLOP | NVIDIA H200 |
| B200 | Blackwell | 192 GB HBM3e | 8 TB/s | 2.25 PFLOPS | 3.6 GB/s per TFLOP | NVIDIA Blackwell |
| B300 (NVL72) | Blackwell | 288 GB HBM3e | 16 TB/s (2-die) | ~4.5 PFLOPS | 3.6 GB/s per TFLOP | NVIDIA Blackwell |
| Rubin (R100) | Rubin | HBM4 (TBD) | ~36 TB/s (est.) | TBD | TBD | NVIDIA Rubin |
The divergence is clear: H100→B200 compute grew ~2.3× while bandwidth grew only ~2.4× — but the arithmetic intensity ceiling (max FLOPs per byte) is what matters for memory-bound inference. MTP directly attacks this ceiling: by reusing the same KV-Cache for k predictions, it effectively multiplies the compute-per-byte ratio by k×, pushing inference workloads into the compute-bound regime where bandwidth headroom exists.
2. Interconnect: The Low-Latency vs High-Bandwidth Fork
MTP does not uniformly benefit interconnects. The impact bifurcates by scale:
| MTP Scale | Production Status | Favors | Disfavors |
|---|---|---|---|
| Small MTP (k=1-2) | Deployed (DeepSeek-V3) | Large supernodes, low-latency semantics (e.g., LPX-class) | — |
| Deep Speculative (DSpark-style) | Deployed (June 2026) | Semi-autoregressive drafting + confidence verification | Naive deep stacking |
| "Large MTP" (k=5+, naive stacking) | Theoretical only — acceptance decay makes it uneconomical | High-bandwidth fabrics (if solved) | Low-latency mechanisms |
Breakdown:
- Small MTP (k=1-2) is the current production reality. At this scale, low-latency interconnects remain valuable — LPX-class SRAM-centric chips still hold advantage for latency-sensitive decode.
- Deep Speculative (DSpark) is DeepSeek's answer to the acceptance decay problem: instead of stacking MTP deeper, it uses semi-autoregressive drafting with confidence-scheduled verification, achieving 60-85% speedup [[5]](#ref-5>. This is the actual production mechanism for "beyond k=2" gains — not naive MTP stacking.
- "Large MTP" (k+ naive stacking) remains theoretical because acceptance rate decays rapidly with depth. The k× compute benefit is real only if acceptance can be solved — which is exactly why DSpark abandoned this path.
The inflection point depends on which MTP regime dominates: small-MTP preserves low-latency value; DSpark-style deep speculative shifts value toward verification-bandwidth.
Industry comparison: NVIDIA's Groq 3 LPX achieves 150 TB/s per LPU via on-chip SRAM — ~6.8× more than Rubin's ~22 TB/s HBM4 [[2]](#ref-2). LPX is explicitly designed for the small-MTP/low-latency decode scenario, paired with Rubin GPU for prefill/attention. This positioning is consistent with MTP's bifurcated impact — LPX serves the low-latency branch that remains valuable.
3. Supernode Domain: From Weight Amortization to Communication Bottleneck
The logic chain:
MTP ↑ → Compute intensity ↑ → Per-node HBM pressure ↓
→ Large EP's "weight amortization" benefit suppressed
→ Bottleneck shifts to communication (large-block transfers)
→ Favors high-bandwidth within small supernode domainsLarge EP's essence: distribute experts across nodes so each node loads only a fraction of weights, amortizing HBM bandwidth. MTP undermines this — if HBM bandwidth is no longer the binding constraint, the communication overhead of large EP becomes unjustified.
Implication: The bottleneck moves to communication, but specifically large-block communication — which favors high-bandwidth fabrics within compact domains (rack-scale), not wide-area interconnects.
4. Specialized Silicon: The SRAM-Only Squeeze
Pure SRAM architectures without GPU partners face market suppression — but LPX is an exception, not the rule.
| Chip | Architecture | SRAM | HBM | Bandwidth | GPU Partner? | MTP Impact |
|---|---|---|---|---|---|---|
| Cerebras WSE-3 | Wafer-scale SRAM | 44 GB on-wafer | None | ~21 PB/s | ✗ | Memory advantage diluted |
| Groq Trillium | Deterministic dataflow | Massive on-chip | None | 80 TB/s | ✗ | SRAM premium harder to justify |
| Etched Sohu | Hardwired transformer ASIC | On-chip weights | None | Extreme | ✗ | Compute-bound friendly |
| NVIDIA LPX (Groq 3) | SRAM-centric LPU | 500 MB/LPU, 128 GB/rack | None | 150 TB/s per LPU | ✓ (Rubin NVL72) | Preserved — serves low-latency branch |
| NVIDIA B200 | HBM-balanced GPU | Minimal | 8 TB/s HBM3E | 8 TB/s | — | Better positioned — compute-first |
The LPX exception: LPX is explicitly designed as Rubin NVL72's decode partner — LPU handles low-latency decode, GPU handles prefill/attention [[2]](#ref-2). This maps directly onto MTP's small-MTP/low-latency branch (Section 2), so LPX's value proposition is reinforced, not eroded. Its claimed 35× throughput-per-megawatt is viable precisely because it targets the latency-sensitive regime where SRAM's advantage persists.
The squeeze targets: Pure SRAM architectures without a GPU partner — Cerebras WSE-3 (~21 PB/s wafer bandwidth) and Groq Trillium (deterministic dataflow, 80 TB/s) — face a harder calculus. Their design assumption is that eliminating off-chip memory is the key optimization. When MTP shifts workloads toward compute-bound, SRAM's bandwidth advantage dilutes while its area/power penalty persists. Cerebras benchmarks show >6× speed advantage over Groq at wafer scale [[7]](#ref-7), but this advantage is measured on memory-bound workloads — it narrows as workloads shift.
The pure-SRAM assumption — that eliminating off-chip memory is sufficient — is breaking. Architectures without a GPU partner (Cerebras, Trillium) face a narrower market window. Those with a GPU partner (LPX) are positioned for the low-latency branch that MTP preserves.
5. On-Chip Media: The Great Equalizer
MTP's compute-centric shift has a democratizing effect on chip supply chains:
- Benefits domestic/alternative storage media: Reduced dependence on extreme HBM bandwidth opens the door for domestic HBM3E, CXL-attached memory, and even advanced DDR configurations.
- Benefits mid-tier process nodes: Compute-bound workloads care less about HBM bandwidth (a premium-process differentiator) and more about raw FLOPs (achievable at 4-5nm).
- Hurts premium HBM bandwidth: The monopoly premium of HBM3E (SK Hynix 62% market share) weakens when bandwidth is no longer the binding constraint.
Everyone gets a seat at the table. The structural shift erodes the HBM ecosystem's pricing power — a multi-year trend reversal.
Data point: HBM3E provides 1.2 TB/s per stack; a mid-tier DDR5 channel provides ~50 GB/s. The 24× bandwidth gap matters less when workloads become compute-bound. The global HBM market is projected at $58B in 2026 [[3]](#ref-3) — MTP won't shrink this market, but it will compress its premium.
6. Compute Density: Accelerated Stacking, Then the Next Wall
Two-phase dynamic:
Phase 1 (1-2 years): With the memory wall temporarily relaxed, compute density stacks faster. Chiplets, 3D packaging, and wafer-scale integration face fewer bandwidth constraints per compute unit.
Phase 2 (2-3 years): Compute doubles → memory becomes the bottleneck again. A new HBM growth cycle triggers.
Memory wall broken → Density stacks → Compute doubles
→ New memory pressure → HBM enters next growth cycleIndustry parallel: This echoes the 2022-2024 cycle where HBM3 (819 GB/s) → HBM3E (1.2 TB/s) → HBM4 (2.0 TB/s per stack) adoption accelerated precisely because GPU compute outpaced memory bandwidth [[4]](#ref-4). MTP compresses this cycle.
7. "Memory Is No Longer a Problem" — A Common Fallacy
The claim that MTP eliminates memory pressure is wrong.
- MTP temporarily reduces memory pressure (same KV-Cache, more compute).
- But compute density grows faster — applications rapidly absorb freed compute.
- In 1-2 years, memory demand re-strengthens — not because MTP failed, but because compute growth outpaces HBM bandwidth growth.
MTP doesn't destroy the memory wall — it delays it to a higher compute baseline.
8. Off-Chip Media: DDR's Counterintuitive Upside
A structural possibility: DDR may actually benefit.
Logic: MTP reduces HBM bandwidth dependency → but model capacity demand grows (longer sequences, larger models). If HBM capacity becomes the constraint, DDR as a capacity tier gains relevance. Inference systems may shift from "HBM-only" to "HBM + DDR" tiered storage.
This is not a certainty, but a structural opening — MTP changes the marginal substitution rate between memory tiers.
9. Token Cost & Sequence Length: The Acceleration
| Dimension | Current (H2 2026) | Projected (2027) |
|---|---|---|
| Inference token cost | Baseline | 2-4× decline |
| Mainstream sequence length | 256K | 2M+ |
MTP directly lowers token cost via better compute utilization. DeepSeek's DSpark (June 2026) claims 60-85% faster per-user generation vs MTP-1 baseline [[5]](#ref-5), with SGLang benchmarks showing 1.4× throughput using MTP-based speculative decoding [[6]](#ref-6).
Long-term trends unchanged, but accelerated:
- Low-latency UB ✓
- High-bandwidth fabrics ✓
- High-capacity HBM ✓
- Ultra-low-latency interconnects ✓
- Extreme-length sequences ✓
These directions don't change — they just get faster.
10. Architecture War: Why Only Sparse Scales
Two paths to million-token sequences:
| Path | Representative | Memory Cost | Compute Cost | Multi-Tier Friendly |
|---|---|---|---|---|
| Linear Attention | Kimi3 (KDA) | Linear (but exponential decay) | Linear + Full Attn | ✗ |
| Sparse Attention (DSA) | DeepSeek V4, GLM52 | Sparse-controllable | Sparse-controllable | ✓ |
Kimi3's fundamental problem: Linear + Full Attention hybrid still implies poor scaling of memory and computational cost. Under current forget-gate mechanisms, linear attention's effective memory capacity decays exponentially with sequence length (forget-gate information loss compounds) — meaning the "linear" claim holds only for short-context quality, not for million-token effective retention. The 25% Full Attention retained by Kimi3 remains quadratic at scale.
Within the current forget-gate paradigm, only sparse attention can scale storage/memory cost sub-quadratically. The compounding information decay in linear attention is a mathematical property of the gated recurrence, not a solvable engineering gap — it can be mitigated (e.g., better gate designs, hybrid ratios) but not eliminated without effectively becoming sparse.
Kimi's own late-2025 technical report acknowledged this: they recognized the need to merge into the sparse path. Kimi3's productization simply chose linear attention first.
Prediction: By H2 2027, most leading models will have converged on the DeepSeek sparse + MTP route.
Summary Matrix
| Dimension | Near-Term (1yr) | Medium-Term (2-3yr) |
|---|---|---|
| Compute chips | Compute/KV ratio rises | Intensity ceiling approaches/exceeds training |
| Interconnect | Fork: small MTP→latency / large MTP→bandwidth | Inflection at MTP scale deployment |
| Supernode | Large EP benefit suppressed | High-bandwidth small domains become core asset |
| Specialized silicon | Streaming chips (LPX) window narrows | SRAM-centric assumptions obsolete |
| On-chip media | Domestic/mid-tier gain access | HBM monopoly premium erodes |
| Compute density | Accelerated stacking | New HBM growth cycle triggered |
| Memory wall | Pressure eased (not eliminated) | Re-strengthens as compute doubles |
| Off-chip media | DDR finds tiered role | Hybrid memory architectures |
| Token cost | 2-4× decline | 2M+ sequences standard |
Final Word
DeepSeek's strategy is not accidental — it represents a systems design methodology: identify the cost structure's most expensive link, attack it with algorithmic innovation, and let the market follow. Compared to Kimi3's "stack precision" approach, DeepSeek's systems-level thinking is in a different league.
By next year or H2 2027, expect most leading models to have converged on the DeepSeek playbook.
References
[1] DeepSeek-V3 Technical Report — MTP mechanism: 1-2 prediction depths, sequential causal chains, 0.1 loss scaling. arXiv:2412.19437 | NVIDIA MTP Docs
[2] NVIDIA Groq 3 LPX Architecture — 500 MB SRAM/LPU, 150 TB/s bandwidth, 128 GB/rack, 35× throughput-per-watt. NVIDIA Blog | Spheron Analysis
[3] HBM Market Data — $58B projected 2026, SK Hynix 62% share. HBM3E: 1.2 TB/s/stack, HBM4: 2.0 TB/s/stack. Introl HBM Evolution | SemiAnalysis
[4] HBM Generations — HBM3: 819 GB/s → HBM3E: 1.2 TB/s → HBM4: 2.0 TB/s per stack. GPU: H100 3.35 TB/s → B200 ~8 TB/s → Rubin ~22 TB/s. Wikipedia HBM
[5] DeepSeek DSpark — 60-85% faster speculative decoding (June 2026). Semi-autoregressive drafting + confidence-scheduled verification; explicitly addresses acceptance decay in deep MTP stacking. AcingAI | arXiv:2607.05147
[7] Cerebras CS-3 vs Groq LPU — >6× speed advantage at wafer scale (memory-bound benchmarks). Cerebras Blog
[6] SGLang Speculative Decoding — 1.4× throughput with MTP on DeepSeek models. HPC-AI Tutorial
Related Reading (This Site)
- Million Sequences: Storage vs Compute — Which Is the Real Bottleneck?
- Kimi3 Architecture Analysis: Linear Attention, Sparse Attention, and the Architecture War
- Kimi3 Cost Efficiency: Why the Linear Path Cannot Scale Cost
Industry Benchmarks Referenced
- Cerebras CS-3 vs Groq LPU — >6× speed advantage at wafer scale
- AI Inference Accelerators Compared — Cross-architecture token/s benchmarks
- Comparing AI Hardware Architectures — SambaNova/Groq/Cerebras vs NVIDIA