Per-user AI token quotas: your backend decides who is counted, and a 429 is not a diagnosis

A per-user token quota rests on three decisions: who the user is, what you count, and who answers when the limit is reached. The first one decides whether the other two mean anything. If the browser or the mobile app can set the user id or the plan name, a caller can change whose quota is charged, and the quota holds only for users who don't try.

Two claims run through this article. First, the gateway counts, but your backend decides who is counted. Authentication and the entitlement lookup belong on your side of the trust boundary, and whatever the gateway receives is an assertion from whoever holds the API key. Second, a 429 tells you that someone refused, not who refused or why. A provider rate limit, a per-user quota and your gateway vendor's own plan limit can all arrive as 429, and each needs a different response from your client.

Everything below that describes expected behavior is a design criterion, not a result from tests we ran. The user ids, plans and limits are fictional.

The gateway counts. Your backend decides who is counted.

The user identity at the gateway is only as trustworthy as the component that set it. There are two sound patterns.

Your backend sets it. The user authenticates to your app. Your backend reads the user id from the verified session, looks up their plan in your billing system, and attaches both to the model call. Any identity or plan header that arrives from the client is dropped, never forwarded.

An identity-aware proxy sets it. Cloudflare's AI Gateway behind Cloudflare Access is one documented example: when a request carries a valid Access JWT, the gateway records the JWT sub claim as cf.user_id. The same page lists the limit of that model: service-token requests carry no cf.user_id, because a service token is not a person. Batch jobs and agents acting for a user have the same gap in every design. Something has to attribute them explicitly.

The pattern to avoid is a client-set header passed through untouched. A gateway can only act on the identity it is handed. If the client controls that header, the gateway enforces the quota faithfully against whatever name the caller chose.

Send a pseudonym, not an email address. Derive it with a keyed hash so nobody can recompute it from a user id alone:

import hashlib
import hmac
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.worksi.ai/v1",
    api_key=os.environ["AIW_API_KEY"],  # server-side only
)
PSEUDONYM_KEY = os.environ["END_USER_PSEUDONYM_KEY"].encode()


def end_user_id(user_id: str) -> str:
    return hmac.new(PSEUDONYM_KEY, user_id.encode(), hashlib.sha256).hexdigest()[:32]


def call_model(session, entitlements, messages):
    # session.user_id comes from your verified session,
    # never from the request body or a client header.
    plan = entitlements.plan_for(session.user_id)  # your billing system
    return client.chat.completions.create(
        model="gpt-5-mini",
        messages=messages,
        extra_headers={
            "X-AIW-End-User": end_user_id(session.user_id),
            "X-AIW-End-User-Plan": plan,
        },
    )

Two consequences follow. A pseudonym is an identifier, not proof of identity: anyone holding the gateway API key can send any value, so the key never ships in a browser bundle or a mobile binary. And rotating the hash key changes every user's identifier, so usage before and after the rotation lands on two different ids. Treat the key as long-lived, and before any rotation find out how your implementation handles a changed identifier for each quota dimension.

Decision table: identity and quota dimensions

Combining scopes is a documented pattern elsewhere. Octelium's AI gateway enforces every matching token limit, which lets one service carry both a per-user and a per-tenant quota, and it recommends a separate request rate limit to catch high-frequency loops.

Tokens, requests and spend measure different things

A token limit is not a spend limit. Anthropic's pricing page, read on 5 November 2026, lists Claude Haiku 4.5 at $1 per million input tokens and $5 per million output tokens, and Claude Sonnet 4.6 at $3 and $15. The same 20,000 tokens cost different amounts depending on the model and on the input and output split. A per-user token cap on a plan that allows several models bounds work, not money.

Cached input makes the gap wider. Anthropic's rate limit documentation explains that input_tokens covers only the tokens after the last cache breakpoint. Its example: a 200k-token cached document and a 50-token question report input_tokens: 50, with the cached part in cache_read_input_tokens. A quota that sums input_tokens lets that user through on almost nothing. A quota that also counts cache_read_input_tokens charges them in full for tokens the same pricing page, read on 5 November 2026, lists at 0.1x the base input rate. Decide which you mean, and find out which fields your gateway reads.

Spend limits have their own catch. Traefik's token quota middleware offers separate input, output and total token limits plus a cost limit in US dollars. Its docs label that cost as an estimate from provider-reported tokens and the middleware's own price list, and send you to the provider's billing dashboard for the real amount. Any per-user spend figure works the same way: the price list is part of the quota, and a stale price is a wrong limit.

In-flight requests: reserve first or settle after

You cannot know a request's token count until the response finishes, so there are two designs, and each leaks differently.

Reserve, then reconcile. Before calling the provider, the gateway holds an estimate: input tokens plus the maximum output the request allows. If the reservation does not fit, the request is refused. After the response, the hold is replaced by reported usage. Octelium documents this design: its token limit "reserves estimated input and maximum output capacity before calling the provider, then reconciles it with provider-reported usage." The cost is false refusals. A user with 3,000 tokens left who sends max_tokens: 4096 is refused even if the answer would have been 200 tokens.

Count after the response. The request goes through if the user is under the limit now, and usage is added when it completes. App AI Gateway documents the consequence for its budgets: cost is known only when the response completes, the counter reads the month's spend at most every ten seconds, and "a single very expensive call, or a burst inside those seconds, can overshoot."

Neither design is wrong. Reservation suits hard caps on expensive plans. Post-response accounting suits generous plans where a small overshoot is cheaper than a false refusal. Testing tells you which one you run.

Then decide what happens when usage never arrives. A client cancels a stream halfway. An upstream times out. A response format carries no usage at all: Traefik's docs say MCP responses carry no token usage data, and the middleware rejects token rate limiting for them rather than guessing. If an implementation records missing usage as zero, the quota undercounts. Test which of four things your implementation does when usage is absent: reconcile later, charge an estimate, keep the reservation, or record zero. Our recommendation under reservation is to keep the hold until real usage replaces it.

Four kinds of 429, and two refusals that are not 429

RFC 6585 defines 429 as "too many requests in a given amount of time", says the response MAY include Retry-After, and leaves open how the server identifies the user and counts requests. So the status code carries almost no diagnosis. These are the causes to separate, using Worksi's codes as the gateway example:

Do not build one retry schedule for all of them. A client that retries every 429 with backoff hammers a spend cap that will not lift for weeks, and tells a quota-limited user to wait when they need an upgrade.

The test matrix

Run these in a staging environment with fictional users: u_free_a on plan free with a fictional 20,000 tokens a day, and u_pro_b on plan pro. Expected results follow this article's design, which is a recommendation, not a standard. Where your implementation differs, record what it does.

The bypass row is the easiest to get wrong in both directions. Any service with a raw provider key, a second gateway route without the quota, or a forgotten staging key can call the model without being counted. But a mismatch between the provider's usage and the gateway's totals has innocent causes too. Before you treat it as a bypass, align the comparison:

• Metric and fields. Compare the same token fields, including cached input, or compare request counts, not tokens on one side and requests on the other.
• Window. Use the same time zone and boundaries, and wait until both the provider's usage reporting and the gateway's telemetry have caught up.
• Telemetry completeness. Confirm the gateway's records for the window are complete. A gateway can keep serving requests while its telemetry is delayed or buffered, so a missing record can be a reporting gap rather than a different route.
• Models and keys. Restrict both sides to the provider keys and models the gateway uses.
• Retries and fallbacks. Distinguish logical client requests from upstream attempts. Check whether the gateway view you compare against includes every retry attempt before comparing counts. A fallback moves usage to another model or provider.
• Estimates. If either side prices from a list, compare tokens before you compare money.

What remains after that is an investigation lead, not proof. A provider request or a credential in use that no gateway record explains tells you where to look. To conclude that traffic bypassed the gateway, you need positive route evidence: application or egress logs showing a direct call to the provider from outside the enforcing gateway, correlated to the provider's request id. The prevention is boring: the gateway is the only holder of provider keys.

Where this sits in Worksi

In Worksi, your backend sends X-AIW-End-User (a pseudonymous id) and X-AIW-End-User-Plan on each gateway call, and an end_user_quota policy sets per-plan limits: requests per UTC day, input plus output tokens per UTC day, cost per calendar month in USD, and allowed models. Calls over a limit get 429 end_user_quota_exceeded. They never reach your provider and are not counted as AI traces against your subscription. The policy can run in monitor mode first, recording what would have been limited without blocking anyone: a cheap way to try these cases on real traffic before you enforce. Read the end-user plans and quotas docs for the policy format.

Worksi does not authenticate your end users. It records whatever id and plan your API key sends, and the docs say the end-user id is stored as it is and not redacted, so never send an email address. When the plan header is missing, the policy's default_plan applies. Spend is priced from your model registry, so the cost limit is only as right as those prices. Our docs do not state how Worksi counts concurrent calls near a limit, cached input tokens or cancelled streams, so the concurrency and missing-usage tests are yours to run. End-user quota policies are included from the Startup plan. The headers and per-end-user usage work on every plan, and a new account's first organization gets 3 days of Scale features, which include end-user policies.

A gateway quota is the wrong tool when the limit is a product rule rather than a usage rule, such as "three reports per month" where one report makes twenty model calls. Count that in your application, where the report exists. Use the gateway quota as the backstop that holds when your application logic has a bug.