"Documentation Index\n\nFetch the complete documentation index at: /llms.txt\n\nUse this file to discover all available pages before exploring further.\n\nSkip to main content\nSGLang Documentation home page\nSearch...\n⌘K\nAsk Assistant\nGet Started\nUser Guide\nHardware\nCookbook\nSGLang Diffusion\nGet Started\nWelcome to SGLang\nInstallation\nQuickstart\nTutorial: Sending a request\nGet Started\nWelcome to SGLang\nCopy page\n\nHigh-performance serving framework for large language and multimodal models.\n\n \n\nPerformance & Runtime\nDesigned for low-latency, high-throughput inference with RadixAttention, prefix caching, and multi-GPU parallelism.\nModels & Ecosystem\nBroad support for Llama, Qwen, DeepSeek, and more. Compatible with Hugging Face and OpenAI APIs.\nExtensive Hardware Support\nNative support across Hardware Platforms including NVIDIA, AMD, Intel Xeon, Google TPU, and Ascend NPU accelerators.\nCommunity & Training\nOpen-source with widespread adoption, powering 400k+ GPUs and integrated with major RL frameworks.\nSGLang powers large-scale production deployments, generating trillions of tokens each day across more than 400,000 GPUs worldwide. It is hosted under the non-profit open-source organization LMSYS.\n\nGet Started\nSGLang is an inference framework meant for production level serving. It is designed to deliver low-latency and high-throughput inference across a wide range of setups, from a single GPU to large distributed clusters.\nInstall SGLang\nInstall SGLang with pip, from source, or via Docker on your preferred hardware platform.\nQuickstart\nLaunch your first model server and send requests in minutes with OpenAI-compatible APIs.\n\nNews and latest blogs\n\nServing GLM5.2 NVFP4 Agentic Workload with SGLang: Reaching 500 TPS in 2 Weeks\n\nJuly 14, 2026\n\nBringing DeepSeek-V4 Flash RL Training to AMD Instinct MI355X GPUs with Miles\n\nJuly 10, 2026\n\nDSpark in SGLang: Speculative Decoding with Confidence-Driven, Variable-Length Verification\n\nJuly 6, 2026\n\nAgent-Assisted SGLang Development: An Initial Exploration\n\nJuly 2, 2026\n\nAccelerating SGLang HiCache with Netpreme X-Mem™ MPU\n\nJuly 8, 2026\n\nImproving DeepEP MoE Load Balance in SGLang with Waterfill and LPLB\n\nJune 26, 2026\n\n\nLearn more and join the community\n\nSTAY CONNECTED\n\n Development roadmap to follow current priorities and upcoming work.\n Weekly public development meeting to hear updates and join open discussions.\n Slack for questions, feedback, and community support.\nX Twitter and \n LinkedIn for project updates.\n LMSYS blog for release notes, benchmarks, and technical deep dives.\n Learning materials for blogs, slides, and videos.\n\nWas this page helpful?\n\nYes\nNo\nInstallation\nNext\n⌘I\ngithub\nx\nlinkedin\nslack\ndiscord\nPowered by\nThis documentation is built and hosted on Mintlify, a developer documentation platform"