"Bỏ qua để đến phần nội dung chính\nHỗ trợ truy cập\nĐăng nhập\nChế độ AI\nTất cả\nHình ảnh\nVideo\nMua sắm\nVideo ngắn\nTin tức\nThêm\nCông cụ\nKết quả tìm kiếm\nLooking for results in English?\nChange to English\nTiếp tục sử dụng Tiếng Việt\nCài đặt ngôn ngữ\nThông tin tổng quan do AI tạo\nFor long-context vLLM workloads, Prefill tasks are compute-bound (need GPU TFLOPS and NVLink), while Decode workers are memory-bound (need raw High Bandwidth Memory or HBM). To scale, use Disaggregated Prefill and Decode so that memory-heavy decode requests do not compete for HBM space and bandwidth. \nSpheron\n +1\nKey Sizing Formula and Parameters\nThe formula to determine the High Bandwidth Memory (HBM) required for a long-context decode worker is:\n𝑀\n𝑒\n𝑚\n𝑜\n𝑟\n𝑦\n𝑡\n𝑜\n𝑡\n𝑎\n𝑙\n=\n(\n𝑊\n𝑒\n𝑖\n𝑔\nℎ\n𝑡\n𝑠\n+\n𝐴\n𝑐\n𝑡\n𝑏\n𝑢\n𝑓\n𝑓\n𝑒\n𝑟\n)\n+\n(\n𝐵\n𝑎\n𝑡\n𝑐\nℎ\n𝑠\n𝑖\n𝑧\n𝑒\n×\n𝐶\n𝑜\n𝑛\n𝑡\n𝑒\n𝑥\n𝑡\n𝑙\n𝑒\n𝑛\n𝑔\n𝑡\nℎ\n×\n𝐿\n𝑎\n𝑦\n𝑒\n𝑟\n×\n𝐻\n𝑒\n𝑎\n𝑑\n×\n𝐷\n𝑖\n𝑚\n×\n𝐵\n𝑦\n𝑡\n𝑒\n𝑠\n𝑝\n𝑒\n𝑟\n_\n𝑝\n𝑎\n𝑟\n𝑎\n𝑚\n×\n2\n)\n(Note: The \"2\" accounts for both Key and Value caches in KV-caching).\nTo prevent OOM (Out-of-Memory) and ensure smooth decoding, vLLM administrators must configure several critical flags:\n--block-size: vLLM divides the KV cache into logical blocks. For long contexts, use a larger block size like 128 to reduce memory fragmentation.\n--max-model-len: Dictates the maximum advertised context length. If planning for 128k context windows, this must be set to 131072.\n--max-num-batched-tokens: Governs chunk size. vLLM chunk-prefills (or splits) large prompts into sizes defined by this flag to keep memory usage stable. \nvllm.ai\n +1\nHardware Planning for Decode Workers\nDecode workers rely entirely on HBM capacity and HBM memory bandwidth. With long contexts, the sheer size of the KV cache can exceed the model weights themselves. \nSpheron\n +2\nRecommended Hardware: The NVIDIA H200 (141GB HBM3e) is built specifically for this. It provides 4.8 TB/s of bandwidth to read the entire KV-cache stack in microseconds for every generated token.\nBudget Option: For smaller models or shorter contexts, the NVIDIA A100 (80GB) is a cost-effective option. \nSpheron\n +1\nHiện thêm\nWeb results\nLong Sequence Context Parallel (Multi Node) - vLLM Ascend\n\nvLLM\nhttps://docs.vllm.ai › tutorials › features\n·\nDịch trang này\nvLLM-Ascend now supports long sequence with context parallel options. This guide takes one-by-one steps to verify these features with constrained resources.\nLLM Serving in Production with vLLM - Ammar Bagasrawala\n\nMedium · Ammar Bagasrawala\n3 lượt thích · 6 tháng trước\nvLLM provides a comprehensive suite of metrics designed to enable robust observability, performance tuning, and capacity planning for large ...\nHow to Deploy an LLM On-Premise (2026)\n\nIternal AI\nhttps://iternal.ai › how-to-deploy-llm-o...\n·\nDịch trang này\n30 thg 5, 2026 — Step-by-step guide to deploying an LLM on-premise in 2026: VRAM/GPU sizing math, model selection, vLLM vs NVIDIA NIM, Intel Xeon CPU ...\n[Performance]: decoding speed on long context #11286\n\nGitHub\nhttps://github.com › vllm › issues\n·\nDịch trang này\n17 thg 12, 2024 — In our experiments, we found that the decoding speed of vLLM decreases dramatically when the length of the prompt becomes longer.\nBị thiếu: HBM capacity\nvLLM Optimization Techniques: 5 Practical Methods to ...\n\nJarvis Labs\nhttps://jarvislabs.ai › blog › vllm-optim...\n·\nDịch trang này\n6 thg 2, 2026 — Learn 5 practical vLLM optimization methods: prefix caching, FP8 KV-cache, CPU offloading, disaggregated prefill/decode, and zero-reload ...\nBị thiếu: HBM | Phải có: HBM\nThe Workload–Router–Pool Architecture for LLM Inference ...\n\narXiv\nhttps://arxiv.org › html\n·\nDịch trang này\nOver the past year, the vLLM Semantic Router project has released a series of work spanning: (1) core routing mechanisms—signal-driven routing, context-length ...\nPrefill-Decode Disaggregation on GPU Cloud: Split LLM ...\n\nSpheron\nhttps://www.spheron.network › blog\n·\nDịch trang này\n3 thg 4, 2026 — Decode nodes need maximum HBM bandwidth and capacity. Each generated token requires loading KV vectors for every prior token from HBM. The ...\nOptimization and Tuning\n\nvLLM\nhttps://docs.vllm.ai › configuration › o...\n·\nDịch trang này\nIt helps achieve better GPU utilization by locating compute-bound (prefill) and memory-bound (decode) requests to the same batch. Performance Tuning with ...\nBị thiếu: HBM | Phải có: HBM\nPrefill Is Compute-Bound. Decode Is Memory-Bound. Why ...\n\nTowards Data Science\nhttps://towardsdatascience.com › prefi...\n·\nDịch trang này\n15 thg 4, 2026 — Prefill workers need high FLOPS but don't need massive HBM capacity. An H100 SXM with 80GB HBM is well-suited. An H200 with 141GB HBM is ...\nMọi người cũng hỏi\nHow to reduce KV cache in vLLM?\nWhy is vLLM so fast?\nWhat is max_num_batched_tokens in vLLM?\nWhat is the GPU ratio for vLLM?\nMọi người cũng tìm kiếm\nMax model len vllm\nPrefix Caching - vLLM\nVllm/vllm-openai\nvLLM Qwen3\nWhisper vLLM\nQwen3 vl vllm\nMax num batched tokens vllm\nKvcache vllm\nĐiều hướng trang\n\t\n1\t\n2\n\t\n3\n\t\n4\n\t\n5\n\t\n6\n\t\n7\n\t\n8\n\t\n9\n\t\n10\n\t\nTiếp\nĐường liên kết ở chân trang\nViệt Nam\nTừ Liêm, Hà Nội - Dựa trên hoạt động trước đây của bạn\n - Cập nhật vị trí\nTrợ giúp\nGửi phản hồi\nQuyền riêng tư\nĐiều khoản"