SGLang

SGLang is an open-source, high-throughput serving framework for LLMs and vision-language models, using RadixAttention for prefix caching and structured generation to deliver low-latency, production-grade inference.