Envoy AI Gateway
Envoy AI Gateway is an open source project for using Envoy Gateway to handle request traffic from application clients to GenAI services. Built on Envoy Proxy, it aims to simplify how application clients interact with Generative AI services, providing a secure, scalable and efficient way to manage LLM and AI traffic with backend rate limiting and policy control. Its stated goals cover resilient connectivity across hosted providers and self-hosted models with automatic failover, observability for performance and cost, enterprise-grade access control and upstream authentication, and an extensible architecture on Envoy. The v1.1 release is current, and the project routes to sixteen providers out of the box: Anthropic, AWS Bedrock, Azure OpenAI, Cohere, DeepInfra, DeepSeek, Google Gemini, Grok, Groq, Hunyuan, Mistral, OpenAI, SambaNova, Tetrate Agent Router Service, Together AI and Vertex AI. Trade-offs to weigh: unlike the hosted gateways already in this catalog, it is Kubernetes-native infrastructure you deploy and operate yourself on top of Envoy Gateway, so it carries real operational cost; the project website publishes no licence text (Apache-2.0 is stated only on the repository); and it is comparatively young, at roughly 2k GitHub stars against the far larger install bases of the incumbent proxies.
