"Skip to content\nNavigation Menu\nPlatform\nSolutions\nResources\nOpen Source\nEnterprise\nPricing\nSign in\nSign up\nsgl-project\n/\nsglang\nPublic\nNotifications\nFork 7.2k\n Star 30.3k\nCode\nIssues\n706\nPull requests\n3.6k\nDiscussions\nActions\nSecurity and quality\nInsights\n[PD] Support PD disaggregation with Prefill PP\n#8846\nMerged\nzhyncs\nmerged 35 commits into\nmain\nfrom\nfixing_pd_pp\n+632\n-82\nLines changed: 632 additions & 82 deletions\nConversation2\n (2)\nCommits35\n (35)\nChecks71\n (71)\nFiles changed11\n (11)\nConversation\nShangmingCai\ncommented\n• \nedited \nCollaborator\nMotivation\n\nTo reduce TTFT, we want to support Prefill PP with PD disaggregation through this PR.\n\nInit commit to support mooncake with PP [PD] Refactor parallel sizes and add pp support for mooncake #8571\nInclude the deepseek PP support from Add pipeline parallelism for DeepSeekV2 #6434\nSupport Prefill PP with PD different TP size.\nComaptible with DP attention and pure dp\n\nDecode PP will be supported in the next PR. @ssssnow\n\nAccuracy Test\nQwen3-8B(Prefill TP 2 PP2, Decode TP 1):\npython sglang/benchmark/gsm8k/bench_sglang.py --port 8000 --parallel 300 --num-questions 300\n100%|█| 300/300 [02:29<00:00, 2.0\nAccuracy: 0.943\nInvalid: 0.000\nLatency: 149.493 s\nOutput throughput: 249.315 token/s\n\nQwen3-8B(Prefill TP 2 PP2, Decode TP 2):\npython sglang/benchmark/gsm8k/bench_sglang.py --port 8000 --parallel 300 --num-questions 300\n100%|█████| 300/300 [00:24<00:00, 12.01it/s]\nAccuracy: 0.947\nInvalid: 0.000\nLatency: 25.186 s\nOutput throughput: 1469.696 token/s\n\nQwen3-8B(Prefill TP 1 PP2, Decode TP 2):\npython sglang/benchmark/gsm8k/bench_sglang.py --port 8000 --parallel 300 --num-questions 300\n100%|█| 300/300 [00:26<00:00, 11.15i\nAccuracy: 0.950\nInvalid: 0.000\nLatency: 27.047 s\nOutput throughput: 1378.700 token/s\n\nQwen3-8B(Prefill DP 2 TP 1 PP2, Decode TP 4):\npython sglang/benchmark/gsm8k/bench_sglang.py --port 8000 --parallel 300 --num-questions 300\n100%|█| 300/300 [00:20<00:00, 14.44i\nAccuracy: 0.950\nInvalid: 0.000\nLatency: 21.158 s\nOutput throughput: 1755.561 token/s\n\nDeepSeek(Prefill TP 8 PP2, Decode TP 16): verified by @ssssnow \npython benchmark/gsm8k/bench_sglang.py --port 9091 --parallel 300 --num-questions 300\nAccuracy: 0.947\nInvalid: 0.000\nLatency: 32.184 s\nOutput throughput: 927.104 token/s\n\nBenchmark & Profiling\nChecklist\n Format your code according to the Code Formatting with Pre-Commit.\n Add unit tests as outlined in the Running Unit Tests.\n Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.\n Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.\n For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.\n Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.\n👍\n3\nShangmingCai and others added 17 commits \n[PD] Refactor parallel sizes and add pp support for mooncake …\n4414782\nrevert false remove …\n4774f60\nfix isort …\nd8dfc02\nfix log error …\n2e639c0\nfix comment …\n940048b\nfix comment …\n8d7bd97\nfix comment …\nb0c5bbf\nfix decode …\n4a51b04\nfix pd pp …\n223b8af\nfix prefill pp schedule issues\ne8e4df9\nupdate mooncake meta …\nd13e0f3\nupdate\nd241c9b\nport from 2aed40c …\n5b9d63d\nMerge remote-tracking branch 'up/main' into fixing_pd_pp\nd610d68\nfix missing import …\n6d55920\nfix layer calculate …\nac563d5\nupdate …\nb591734\n ShangmingCai requested review from ByronHsu, Ying1123, hnyls2002, merrymercy and xiezhq-hermann as code owners \ngemini-code-assist Bot\ncommented\nContributor\n\nWarning\n\nYou have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!\n\nShangmingCai and others added 3 commits \nfix lint …\nbfe2dac\nfix codespell …\n1d2e0ff\nFixing pd pp fork (#8858)\nVerified\n9b95d4f\n ShangmingCai requested review from ispobock and zhyncs as code owners \nssssnow\ncomme"