Text Generation Inference
Open Source
Library
Production LLM serving toolkit
About
Text Generation Inference (TGI) is Hugging Face's production-grade toolkit for serving open LLMs, with tensor parallelism, continuous batching, quantization, and an OpenAI-compatible API for high-throughput deployment.
Compatibility
Supported Languages
python
rust
Details
- Category
- Library
- License
- Apache-2.0