"Bỏ qua để đến phần nội dung chính\nHỗ trợ truy cập\nĐăng nhập\nChế độ AI\nTất cả\nMua sắm\nHình ảnh\nVideo\nVideo ngắn\nTin tức\nThêm\nCông cụ\nKết quả tìm kiếm\nLooking for results in English?\nChange to English\nTiếp tục sử dụng Tiếng Việt\nCài đặt ngôn ngữ\nThông tin tổng quan do AI tạo\nPrefill-Decode (PD) disaggregation splits an LLM's prompt-processing (Prefill) and token-generation (Decode) into separate, independent pools of GPUs. Prefill nodes handle heavy math, while Decode nodes stream tokens. This split relies on KV cache (the saved memory of past conversations) being transferred and precisely sized in HBM (ultra-fast High Bandwidth Memory). \nEmergent Mind\n +1\nBreaking Down the Concepts\nPD Disaggregation: Separates the two distinct jobs in AI generation. Prefill wants high math power (compute-bound), while Decode needs fast data-retrieval to generate tokens sequentially (memory-bound). \narXiv\n +1\nDecode: The step-by-step generation of the answer. It reads the growing KV cache. \nkuncoro.io\nKV Cache Pool: A system like PagedAttention that manages where the Key and Value cache states are stored to prevent wasted memory.\nSizing & HBM: You must balance how much HBM is dedicated to holding active tokens versus generating new ones. HBM is expensive and limited, so managing its size properly determines how many users an LLM can serve at once. \narXiv\n +1\nHow the Process Works\nDecode Node: Works on generating tokens sequentially. As it grows, the memory footprint increases. \nkuncoro.io\nKV Cache Offloading: If the decode HBM fills up, systems (like vLLM) move cold, unused KV cache blocks from fast HBM to cheaper, slower CPU or host memory (DRAM). \nvLLM\n +1\nDisaggregated Hand-off: When a user re-prompts, the prefill instance fetches the cached KV states from a distributed KV pool and handles the prompt, then passes the results over to the Decode node via fast RDMA (remote direct memory access). \nvLLM\n +1\nHow to Calculate and Size Your KV Cache Pool\nProper KV cache sizing ensures you do not run out of HBM space and crash your server. The formula to estimate the HBM space (S) in gigabytes needed for a single sequence's KV cache is:\nHiện thêm\nWeb results\n[Roadmap] Prefill-Decode Disaggregation Roadmap (2026 ...\n\nGitHub\nhttps://github.com › sglang › issues\n·\nDịch trang này\n30 thg 3, 2026 — This feature will allow us to send kvcache from prefill HBM to decode host memory when enabling hisparse for NSA models (DeepSeek V3.2/GLM 5).\nDemystifying the Design Space and Best Practices ...\n\narXiv\nhttps://arxiv.org › html\n·\nDịch trang này\n30 thg 6, 2026 — A concrete example illustrates the issue. Consider a PD-disaggregated deployment in which prefill and decode use different KV-cache formats.\nPD Disaggregation - SGLang Documentation\n\nSGLang Documentation\nhttps://docs.sglang.ai › pd_disaggrega...\n·\nDịch trang này\nWhen prefill and decode use different tensor parallelism (TP) sizes (e.g., prefill TP=4, decode DP attention with TP=1), the KV cache memory layout differs ...\nBị thiếu: HBM | Phải có: HBM\nServing Agentic Workloads at Scale with vLLM x Mooncake\n\nvLLM\nhttps://vllm.ai › blog › 2026-05-06-m...\n·\nDịch trang này\n6 thg 5, 2026 — Figure 3: PD disaggregation combined with the distributed KV cache pool via MultiConnector. Prefill: The prefill instance prepares KV blocks ...\nPrefill-decode disaggregation | LLM Inference Handbook\n\nBentoML\nhttps://bentoml.com › llm › prefill-de...\n·\nDịch trang này\nDisaggregate prefill and decode for better parallel execution, resource allocation, and scaling.\nBị thiếu: HBM | Phải có: HBM\nPD Disaggregation in LLM Inference\n\nEmergent Mind\nhttps://www.emergentmind.com › topics\n·\nDịch trang này\n6 thg 12, 2025 — PD Disaggregation separates prefill and decode stages in LLM inference, enabling enhanced throughput, reduced latencies, and optimized ...\nInference Basics: KV Cache, Batching & Parallelism | by s09g\n\nMedium · s09g\n1 lượt thích · 4 tuần trước\nPD disaggregation: separating prefill and decode. Since prefill and decode have different resource profiles, we can place them on different GPUs ...\nSeparating Prefill and Decode for GPU Throughput\n\nTechnoLynx\nhttps://www.technolynx.com › post\n·\nDịch trang này\n3 ngày trước — SGLang PD disaggregation splits prefill and decode into independently scaled GPU pools. How the mechanism works, its KV-cache cost, and when ...\nPrefill-Decode Disaggregation in SGLang: Why Splitting LLM ...\n\nkuncoro.io\nhttps://kuncoro.io › blog › prefill-deco...\n·\nDịch trang này\n1 thg 2, 2026 — Decode is memory-bandwidth-bound — each step is mostly fetching the growing KV cache from HBM for attention, not performing heavy computation.\nMọi người cũng tìm kiếm\nPD disaggregation vLLM\nSGLang pipeline parallel\nSGLang context parallel\nSGLang Roadmap\nSGLang radix cache\nĐiều hướng trang\n\t\n1\t\n2\n\t\n3\n\t\n4\n\t\n5\n\t\n6\n\t\n7\n\t\n8\n\t\n9\n\t\n10\n\t\nTiếp\nĐường liên kết ở chân trang\nViệt Nam\nHà Nội - Dựa trên hoạt động trước đây của bạn\n - Cập nhật vị trí\nTrợ giúp\nGửi phản hồi\nQuyền riêng tư\nĐiều khoản"