vLLM Unified Architecture Report

一份文档,合并了此前所有报告: spine + L1–L4 agent + 第一性原理(软件+物理双视角) + Performance Handbook + Ascend 官方设计文档 + community 90 天 pulse。 每个 section 顶部标注 source: 溯源锚点。全部原始报告保存在 /vllm_research/vllm_analysis_raw/,1:1 可回溯。 阅读顺序: §0 Overview → §1 软件架构第一性原理 → §2 分层 → §3 热点路径 → §4–§10 深度 → §11 引用。

0. Overview & v0→v1 Synthesis

This single document merges all prior analyses (spine, L1–L4 agents, first-principles software+physical, performance handbook, community pulse, Ascend official design docs) into one navigable report without losing any file:line source anchor. All original raw reports are preserved in /vllm_research/vllm_analysis_raw/ for 1:1 traceability. Each section carries a source band (source: …). Reading order: this overview → §1 first-principles → §2 architecture layers → §3 execution hot path → §4–§10 deep dives → §11 references.

0.1 v0 的两条缠结 → v1 的解法

1. Software-Architecture First Principles

1.1 The axiom applied — rates → modules

1.2 Each module — its ONE job and what it never touches

1.3 Cross-module contracts — why they are cut this way

1.4 First-principles meta-judgement — forced vs chosen

Among all vLLM's layers, a few are forced by physics: paged-KV, Attention-as-separable-backend, Communication-via-Connector, Platform-as-interface — these are stable across HW generations. Others are engineering choices a v2 may freely redesign: the Python front-end, the shm-RPC executor, capability-dispatched sampling, the patch layer. Every heavily-patched subsystem is a gap in vLLM's abstraction surface.

2. Architecture Layers

2.1 EngineCore — the time-line owner

v1/engine/core.py:101 EngineCore. The heartbeat (core.py:576):

def step(self):
    scheduler_output = self.scheduler.schedule()          # 1. decide
    future = self.executor.execute_model(scheduler_output, non_block=True)  # 2. async dispatch
    model_output = future.result()                        # 3. wait
    engine_core_outputs = self.scheduler.update_from_output(scheduler_output, model_output)

EngineCore also owns: add/abort, pause/resume, sleep/wake, LoRA hot-plug, cache reset (reset_prefix_cache/reset_encoder_cache), DP coordination (step_with_batch_queue), spec-draft collection (post_step → take_draft_token_ids). It holds NO device/model/tensor — runs CPU-only, the ultimate control/data-plane split.

2.2 Scheduler — the pure decision function

v1/core/sched/scheduler.py:418. The design axiom (scheduler.py:420):

# There's no "decoding phase" nor "prefill phase" in the scheduler.
# Each request just has num_computed_tokens and num_tokens_with_spec …
# At each step, the scheduler tries to assign tokens to the requests so
# that each request's num_computed_tokens can catch up its
# num_tokens_with_spec. This is general enough to cover chunked prefills,
# prefix caching, speculative decoding, and "jump decoding."

Unified token-budget loop over RUNNING then WAITING, chunked via long_prefill_token_threshold, preemption when allocate_slots returns None. Holds no device/model/comm.

2.3 Worker — the device owner

v1/worker/worker_base.py+gpu_worker.py. One Worker per GPU/NPU. Responsibilities: init_device, load_model/reload_weights, determine_available_memory, kv_caches tensors, sleep/wake, LoRA add/remove/pin, execute_model, sample_tokens, health check. Does NOT hold Scheduler, makes no scheduling decisions, holds no request state — consumes SchedulerOutput statelessly. Worker holds ModelRunner but the two are separate: same device can run different models; same model can run on different devices (Ascend NPUWorker(NPUModelRunner) subclasses but keeps the interface).

2.4 ModelRunner — the forward executor

2.5 AttentionBackend — the only HW-sensitive op

Attention is the ONLY op whose memory-access pattern depends on sequence length + KV layout (GEMM/Norm/MoE/RoPE are all regular (N,d)→(N,d)). The triple: register_backend + Builder.build(common) + Impl.forward + KVCacheSpec family (FullAttentionSpec / MLASpec / RSWASpec / RaggedSpec). ModelRunner only sees KVCacheSpec, not concrete model — this decouples "model structure change" from "KV management".

2.6 Platform — device as interface

platforms/. 6 mandatory methods. Holds device capability/memory/stream/attn-backend-selection/graph-pool. Ascend NPUPlatform overrides only identity+backend-map+memory+compile hooks; its get_device_total_memory raises NotImplementedError (torch_npu global-init-only) — a canonical "interface promise vs device capability" mismatch.

2.7 Compilation — "record once, replay many"

compilation/+v1/worker/gpu/cudagraph_utils.py. Platform injects ACLGraphWrapper via get_static_graph_wrapper_cls(). See §6 for the 3-mode trade-off and ACL-specific constraints.

3. Execution Hot Path (Scheduler → ModelRunner → Sample)

3.1 The page-table walk (slot_mapping kernel)

v1/worker/gpu/block_table.py:283 (_compute_slot_mappings_kernel)
block_indices  = positions // (block_size * CP_SIZE)
block_offsets  = positions %  (block_size * CP_SIZE)
block_numbers  = tl.load(block_table_ptr + req_state_idx*stride + block_indices)
slot_ids = block_numbers * block_size + block_offsets   <- physical address

3.2 The 3-line heartbeat

def step(self):
    scheduler_output = self.scheduler.schedule()
    future = self.executor.execute_model(scheduler_output, non_block=True)
    engine_core_outputs = self.scheduler.update_from_output(scheduler_output, future.result())

3.3 SchedulerOutput — the Scheduler↔Worker contract

v1/core/sched/output.py: scheduled_new/resumed/running/preempted reqs, req_to_new_blocks, num_scheduled_tokens, scheduled_spec_decode_tokens, scheduled_encoder_inputs, finished_req_ids. Carries ONLY decisions (block ids + token counts) — no tensors — so it crosses process/node boundaries (PD disagg relies on this). 4 explicit categories (new/resumed/running/preempted) let Worker use simple if-else. Block allocation done in Scheduler side — Worker only consumes block ids.

3.4 ForwardContext — the model↔framework firewall

forward_context.py:132: attn_metadata[per-layer], slot_mapping[per-layer], dp_metadata, cudagraph_runtime_mode, batch_descriptor, is_padding, skip_compiled, +all_moe_layers/moe_layer_index (torch.compile string workaround). Model code (200+ files) reads only ForwardContext — changing scheduling/graph-mode/parallelism never touches model files.

3.5 allocate_slots → None — control-flow unification

while True:
    new_blocks = kv_cache_manager.allocate_slots(request, num_new_tokens)
    if new_blocks is not None: break
    preempted = self.running.pop()   # reclaim budget
    self._preempt_request(preempted)

Returning None (not raising) keeps preempt logic in the same while-loop as allocate logic.

4. KV-Cache Management — why 4 layers, not 1

4.1 BlockPool — pure metadata, content-hash

v1/core/block_pool.py:162: blocks = [KVCacheBlock(idx) for idx in range(num_gpu_blocks)], free_block_queue, cached_block_hash_to_block = BlockHashToBlockMap(). KVCacheBlock (kv_cache_utils.py:117) is pure metadata — no tensor. The actual bytes live in Worker-side kv_caches: list[torch.Tensor] (model_runner.py:490). This two-layer split lets prefix-cache/swap/sliding-window/compressed-MLA all renumber the same block_ids without touching the pool.

4.2 Coordinator subclasses

UnitaryKVCacheCoordinator (single group), HybridKVCacheCoordinator (MLA+GQA mixed, e.g. DeepSeek-V3), KVCacheCoordinatorNoPrefixCache. MLA's storage_block_size ≠ block_size mapping is encapsulated here — Manager only calls Coordinator's interface.

4.3 Why 4 layers — the MLA forcing function

v0 had a single BlockSpaceManager. MLA's storage_block_size ≠ block_size added a logical→physical mapping that would have scattered across allocate/free/prefix-cache. After the split, this mapping is encapsulated in Coordinator. "Multi-group mapping" changes are isolated to one layer.

Splitting manager↔coordinator: new KV layout (MLA) changes Coordinator only. Splitting coordinator↔blockpool: prefix-cache/eviction changes BlockPool only. Splitting manager↔connector: PD disagg/offload changes Connector only.

5. Parallel & Distributed (TP/PP/DP/EP/CP/EPLB)

These compose (production runs TP8+EP64+PP4+DP32 simultaneously). Scheduler does NOT know TP/EP — its granularity is "request", not "token on which card". Worker handles via model-internal ColumnParallelLinear/MoEDispatch; ModelRunner never calls comm directly. Ascend HCCL via distributed/pyhccl.py PyHcclCommunicator.

5.1 Context Parallel (PCP/DCP)

cp_size = pcp_size × dcp_size; token-interleave storage via cp_kv_cache_interleave_size (default 1); block_size % cp_kv_cache_interleave_size == 0 required.

5.2 EPLB — Ascend's original MoE contribution

Expert-migration stack: EplbUpdator → policy (FlashLB Hungarian + numba tree-search, FlashTree, SwiftBalance) → D2DExpertWeightLoader → VllmEplbAdaptor. The only net-new subsystem from Ascend to upstream vLLM.

5.3 PD disagg — cutting the Worker boundary, no new module

P node owns Scheduler+Worker(P)+KVConnector(P→D); D node owns Scheduler+Worker(D)+KVConnector(D←P). Scheduler only sees KVCacheBlocks (ids), not which node holds the block — "request lifecycle ⟷ KV physical location" decoupling, the direct payoff of KVCacheManager being independent of Scheduler.

6. Compilation & Graph (CUDA/ACL Graph, Fusion, Quant)

6.1 Three execution modes

Dispatch: v1/worker/gpu/cudagraph_utils.py CudagraphDispatcher, candidate key (rounded_num_tokens, num_active_loras); each recorded at warm-up, runtime looks up nearest case.

6.2 ACL Graph — why Ascend cannot just port CUDA Graph

Source: official docs/source/developer_guide/Design_Documents/ACL_Graph.md

Diff A — full replay needs backend-side update_graph_params() hook (NPU-only). Capture phase records per-graph task handle/event/workspace; replay frontend calls update_full_graph_params() → backend runs parameter refresh on update stream via torch.npu.graph_task_update_begin/end(); ExternalEvent enforces ordering. Key fact: an attn backend WITHOUT this hook CANNOT do full-graph replay. Implemented in vllm_ascend/attention/attention_v1.py, mla_v1.py, context_parallel/attention_cp.py, mla_cp.py.