Rate Limiting

Reference for local and remote rate limiting policies in Agentgateway.

Agentgateway supports two rate limiting modes:

• Local rate limiting (localRateLimit) — token-bucket rate limiting applied in-process. State is not shared across instances.
• Remote rate limiting (remoteRateLimit) — rate limiting delegated to an external Envoy Rate Limit Service compatible gRPC server. State is shared across all gateway instances.

Both policies are configured under binds[].listeners[].routes[].policies.

Local rate limiting

Local rate limiting uses a token-bucket algorithm. Each route maintains its own bucket in memory.

localRateLimit · object[]

A list of token-bucket rate limit configurations applied to the route. State is kept local to the process.

Show localRateLimit fields

localRateLimit[].maxTokens · integer · required

The maximum number of tokens the bucket can hold. This is also the initial number of tokens when the gateway starts.

maxTokens: 10

localRateLimit[].tokensPerFill · integer · required

The number of tokens added to the bucket on each fill interval.

tokensPerFill: 1

localRateLimit[].fillInterval · string · required

How often the bucket is refilled. Accepts duration strings such as 60s, 1m, 500ms.

fillInterval: 60s

localRateLimit[].type · string

Optional type qualifier for the rate limit rule.

Local rate limiting example

binds:
- port: 3000
  listeners:
  - routes:
    - policies:
        localRateLimit:
        - maxTokens: 10
          tokensPerFill: 1
          fillInterval: 60s
      backends:
      - mcp:
          targets:
          - name: everything
            stdio:
              cmd: npx
              args: ["@modelcontextprotocol/server-everything"]

In this example, the route allows a burst of up to 10 requests, then refills 1 token every 60 seconds (1 request/minute steady-state).

Remote rate limiting

Remote rate limiting delegates rate limit decisions to an external gRPC service compatible with the Envoy Rate Limit Service API. The gateway sends descriptor entries to the service, which returns allow/deny decisions.

remoteRateLimit · object

Rate limit incoming requests using an external rate limit service.

Show Backend address (one required)

remoteRateLimit.host · string

Hostname or IP address (with port) of the rate limit service.

remoteRateLimit:
  host: "127.0.0.1:8081"

remoteRateLimit.service · object

Reference to a named service for the rate limit backend.

remoteRateLimit.backend · string

Explicit backend reference. The backend must be defined in the top-level backends list.

Show remoteRateLimit fields

remoteRateLimit.domain · string · required

The rate limit domain. Must match the domain configured in your rate limit service.

domain: "agentgateway"

remoteRateLimit.descriptors · object[]

A list of descriptor sets sent to the rate limit service with each request. The service uses descriptor entries to look up the applicable rate limit.

Show descriptor fields

descriptors[].entries · object[] · required

A list of key-value descriptor entries.

Show entry fields

entries[].key · string · required

The descriptor key.

entries[].value · string · required

The descriptor value. Can be a CEL expression that evaluates at request time (e.g. '"echo"' or a variable reference).

descriptors[].type · string

Optional type label for the descriptor set (e.g. "requests").

remoteRateLimit.failureMode · string · default: "failClosed"

Behavior when the remote rate limit service is unavailable or returns an error.

• failOpen — allow requests through when the service is unreachable.
• failClosed — deny requests with a 500 status when the service is unreachable.

Defaults to failClosed.

failureMode: failOpen

remoteRateLimit.policies · object

Backend connection policies for the rate limit service (TLS, headers, etc.).

Remote rate limiting example

Gateway config

binds:
- port: 3000
  listeners:
  - routes:
    - policies:
        remoteRateLimit:
          domain: "agentgateway"
          host: "127.0.0.1:8081"
          failureMode: failOpen
          descriptors:
          - entries:
            - key: "user"
              value: '"test-user"'
            - key: "tool"
              value: '"echo"'
            type: "requests"
      backends:
      - mcp:
          targets:
          - name: everything
            stdio:
              cmd: npx
              args: ["@modelcontextprotocol/server-everything"]

Rate limit service config

The Envoy Rate Limit Service configuration that corresponds to the descriptors above:

domain: agentgateway
descriptors:
- key: user
  value: "test-user"
  descriptors:
  - key: tool
    value: "echo"
    rate_limit:
      unit: minute
      requests_per_unit: 5
- key: tool
  value: "echo"
  rate_limit:
    unit: minute
    requests_per_unit: 20

In this configuration, user test-user calling tool echo is limited to 5 requests/minute. All users calling echo share a limit of 20 requests/minute.

The remote rate limit service must implement the Envoy Rate Limit Service gRPC API. The host value must include the port (e.g. 127.0.0.1:8081).

Local rate limit state is not shared across multiple Agentgateway instances. Use remoteRateLimit for distributed deployments where consistent rate limiting across instances is required.