Google Cloud has added managed AI model routing to API Gateway, giving developers one OpenAI-compatible endpoint for models hosted through Vertex AI Model Garden. The feature is now in public preview and can direct requests to Gemini, Anthropic Claude, and OpenAI GPT-family models without requiring teams to operate a separate proxy service, according to Google's announcement.
Routing moves into the gateway
Developers define virtual model names and backend targets inside an OpenAPI 3.x specification. An application then sends a standard chat-completions request to API Gateway, which inspects the model field, matches it against the configured rules, translates the payload into the destination model's native Vertex AI schema, and returns the response through the same client interface, as described in the technical documentation.
That puts model selection alongside familiar gateway controls rather than inside application code. Google says the service can provide centralized authentication, quotas, token tracking, and traffic monitoring while removing the need to host and scale an open-source routing proxy. It can also be paired with the Gemini Enterprise Agent Platform, with Agent Gateway handling agent governance before requests reach API Gateway for model selection, according to the launch post.
Useful abstraction, narrow preview
The preview is more constrained than the phrase "multi-model router" might suggest. Routing currently depends only on the model name supplied in the request. It does not automatically choose a model based on prompt complexity, price, latency, or output quality, according to Google Cloud's documentation.
All models attached to one router must also share the same Vertex AI hostname. The gateway can select different model paths on aiplatform.googleapis.com, for example, but it cannot send traffic to unrelated provider hosts. Public-preview support is limited to text prompts in OpenAI-compatible JSON, while request streaming, gRPC, WebSockets, Gemini Live, and VPC Service Controls are not supported. Server-sent response streaming is supported, according to the documented limitations.
There are deployment constraints as well. A gateway cannot be converted between routed and non-routed modes after deployment, so changing that behavior requires a new API configuration and gateway instance. OpenAPI 2.0 specifications are not supported, and routed operations cannot be mixed with standard gateway operations in the same specification, according to Google's configuration overview.
The result is less an intelligent model picker than a managed compatibility and policy layer. For teams already using Vertex AI Model Garden, it reduces integration differences among model families and centralizes traffic controls. Choosing which model should handle a request remains the developer's job.
