> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/agentgateway/agentgateway/llms.txt
> Use this file to discover all available pages before exploring further.
# Architecture
> How Agentgateway works as a data plane for agentic AI — from configuration hierarchy to zero-downtime updates via xDS.
Agentgateway is a high-performance proxy written in Rust that acts as the data plane for agentic AI connectivity. It sits between AI agents and their upstream targets — MCP servers, A2A agents, or HTTP services — adding security, observability, and governance to every interaction without requiring changes to your agents or tools.
## Configuration hierarchy
Agentgateway uses a four-level hierarchy to describe how traffic is received and forwarded:
```
Binds
└── Listeners
└── Routes
└── Backends
```
A **Bind** maps to a TCP port the proxy listens on. Each bind can host multiple listeners, one per virtual host or protocol.
A **Listener** defines hostname matching, protocol selection (HTTP, MCP, A2A), and optional TLS termination within a bind.
A **Route** specifies match criteria (path, headers, method) and the policies to apply before traffic reaches a backend.
A **Backend** is an upstream target — an MCP server, an A2A agent, an HTTP service, or an AI provider.
This hierarchy maps directly to the configuration schema:
```yaml theme={null}
binds:
- port: 3000
listeners:
- hostname: "*.example.com" # Listener
protocol: http
routes:
- matches:
- path:
pathPrefix: /api
policies:
auth: ...
backends:
- host: api-service:8080
```
The configuration hierarchy intentionally mirrors the Kubernetes Gateway API resource model: `Gateway` (Bind) → `HTTPRoute` → `BackendRef`. This makes Kubernetes deployments feel natural.
## Configuration types
Agentgateway supports three forms of configuration that serve different roles in the system.
**Static configuration** is set exactly once early in the process lifecycle. It covers global settings like logging format, admin server address, connection pool sizes, tracing endpoints, and DNS resolver behavior. Routing, policies, and backends are **not** configurable here.
Static config is provided via:
* Environment variables
* A YAML or JSON file passed to the `--config` flag
* Inline bytes on the command line
```yaml theme={null}
config:
logging:
format: json
level: info
adminAddr: "0.0.0.0:9901"
statsAddr: "0.0.0.0:9902"
tracing:
otlpEndpoint: "http://otel-collector:4317"
```
Because static config is loaded once at startup, changes require a process restart.
**Local configuration** is a YAML or JSON file that defines the full feature set of Agentgateway: binds, listeners, routes, backends, and policies. It uses a **file watch** to dynamically reload changes without restarting the process.
When a file change is detected, the local config is re-parsed and translated into the shared Internal Representation (IR). The proxy then applies the new configuration with zero downtime.
```yaml theme={null}
binds:
- port: 3000
listeners:
- routes:
- policies:
cors:
allowOrigins:
- "*"
backends:
- mcp:
targets:
- name: everything
stdio:
cmd: npx
args: ["@modelcontextprotocol/server-everything"]
```
Local config is convenient for development and standalone deployments. For production at scale, consider XDS.
**XDS configuration** enables the proxy to be configured by a remote control plane over the [xDS Transport Protocol](https://www.envoyproxy.io/docs/envoy/latest/api-docs/xds_protocol). This is the mechanism used when running under the Kubernetes controller.
Unlike local config, XDS does not perform side-effectful operations such as fetching JWKS from URLs or reading files. It is optimized for efficiency and simplicity over human ergonomics.
XDS config uses purpose-built protobuf types rather than Envoy's Listener/Cluster types, and maps directly to the same IR as local config.
```yaml theme={null}
config:
xdsAddress: "grpc://control-plane:18000"
xdsAuthToken: "${XDS_TOKEN}"
```
## Internal Representation (IR)
Both local config and XDS config translate into a shared **Internal Representation (IR)** that the proxy uses at runtime. The IR is an in-memory data structure that the request processing pipeline reads directly.
```
Local config file ──┐
├──► IR ──► Request processing
XDS control plane ──┘
```
The IR translation layer handles ergonomic differences between user-facing configuration and runtime needs:
* **Local config** may fetch JWKS from URLs, resolve file paths, and perform other I/O during translation.
* **XDS config** performs only simple, mechanical remappings — no side effects.
* **In some cases** the IR and local config are identical; in others there are trivial field remappings.
The design goal is a nearly direct mapping from user-facing APIs → XDS → IR. This simplifies operations (configuration is easy to understand), reduces control plane complexity, and enables efficient delta updates.
## xDS and zero-downtime updates
Agentgateway uses xDS as its dynamic configuration protocol, but with purpose-built resource types instead of Envoy's types. This enables several important performance properties:
**Fine-grained resource cardinality.** Each logical object in the user API maps to exactly one protobuf resource:
| User API object | xDS resource |
| -------------------- | -------------------------------------------------- |
| One `HTTPRoute` rule | One `Route` resource |
| One Kubernetes `Pod` | One `Workload` resource |
| One policy | One policy resource targeting its attachment point |
This contrasts with Envoy, where changing one route requires resending all routes for a listener — potentially megabytes of data per update. In Agentgateway, changing one route sends one small protobuf message.
**Policy merging at runtime.** Policies can be attached at the Gateway, Listener, Route, or RouteRule level. Rather than flattening policies to the lowest level (which causes fanout), the control plane sends policies as-is with references to their attachment point. Precedence and merging are resolved at runtime.
**Example:** A user changing a single global TLS cipher suite setting in Envoy would require updating every Cluster resource. In Agentgateway, this is a single `Policy` targeting a `Bind` — one small message, regardless of how many routes or backends exist.
## Kubernetes controller
When deployed on Kubernetes, Agentgateway includes a built-in controller that:
* Watches Kubernetes Gateway API resources (`Gateway`, `HTTPRoute`, `ReferenceGrant`, etc.)
* Translates them into Agentgateway's xDS resource types
* Pushes updates to the proxy over the xDS Transport Protocol
* Handles zero-downtime rollouts and configuration changes
The Kubernetes controller uses the same xDS path as any external control plane, so the proxy behavior is identical in both cases.
```
Kubernetes API server
│
▼
Agentgateway controller (watches Gateway API resources)
│ xDS gRPC
▼
Agentgateway proxy (data plane)
│
▼
Upstream backends (MCP servers, A2A agents, HTTP services)
```
Deep dive into config file structure, file-watch reloads, and the full configuration schema
Learn how Agentgateway proxies MCP, A2A, and HTTP traffic