An MCP session 404 is not proof that the load balancer is at fault

A client logs 404 Session not found from an MCP Streamable HTTP endpoint behind a load balancer, and the first fix someone proposes is sticky sessions. Sometimes that is right. But under the 2025-11-25 revision, a 404 is the response a server owes for any session it no longer recognizes, whatever the reason. The 2025-11-25 transport specification says a server "MAY terminate the session at any time, after which it MUST respond to requests containing that session ID with HTTP 404 Not Found." Routing to the wrong replica is one way to get there. An idle timeout, a restart, a request that reached the wrong endpoint and a client that never reinitialized are others. Affinity fixes only the first.

This article proposes a diagnostic matrix: read-only comparisons that tell those causes apart, each result marked as supporting, contradicting or inconclusive. It is our synthesis from the specification and public reports, not a case from a production system we ran.

Read the whole 404 before you name a cause#

Start with what the status code leaves out. For each failing request, record:

• The negotiated protocol version. The protocolVersion in the InitializeResult, and the MCP-Protocol-Version header sent afterwards. The session rules here are for 2025-11-25.
• The endpoint path. The spec requires a single MCP endpoint path for POST and GET. A proxy that rewrites the path changes what answers.
• The method. A GET answered with 405 means the server offers no SSE stream at that endpoint. That is not a session error.
• Whether the request carried MCP-Session-Id. The session rule is about a 404 on a request that carried the header. A 404 on an InitializeRequest, which carries none, means something else: the spec's backwards-compatibility section tells clients to treat 400, 404 or 405 on initialize as a sign the URL may point to a server running the older HTTP+SSE transport.
• The response body. A JSON-RPC error from your MCP server and an HTML page from an ingress controller are different evidence with the same status.
• Which hop answered. A proxy that keeps one session with the client and another with the upstream can return its own 404 or pass on the upstream's.

Then add time and identity at every hop: a UTC timestamp, the request ID each hop already assigns, the replica, and a process identity that changes on restart, such as a process start time. A pod name can stay the same while the process behind it, and its memory, is new. Put rollouts, restarts and idle gaps on the same timeline, and record when each hop sent a request and received each response: row 5 below depends on that order.

Correlate sessions without logging them#

You need to know that the initialize on replica A and the failing ping on replica B belong to one session. You do not need the session ID in your logs to know it. The spec says the client "MUST handle the session ID in a secure manner". An ID in a log that a dozen people can read is not handled securely.

Log a keyed fingerprint instead: HMAC-SHA256 of the session ID under a key from your secret store, truncated to 16 hex characters. Every hop that holds the key computes the same value, and a reader of the log cannot turn it back into a usable ID. Keep Authorization headers out of the same logs.

Here is a probe that sends ping only after a correlated InitializeResult, a negotiated 2025-11-25, a session ID and an accepted notifications/initialized:

# Read-only session probe for a 2025-11-25 Streamable HTTP endpoint.
# Run only where you are authorized, with the identity your client normally uses.
# Needs curl, jq, openssl. Parses JSON responses only: an SSE reply is inconclusive.
set -euo pipefail
umask 077
URL="https://mcp.staging.example.com/mcp"   # the exact path your client uses
TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT

post() {  # post NAME JSON [curl args]; prints the HTTP status, 000 on timeout
  local n="$1" body="$2"; shift 2
  curl -sS --connect-timeout 5 --max-time 15 -o "$TMP/$n.body" -D "$TMP/$n.hdrs" \
    -w '%{http_code}' -H "Authorization: Bearer $MCP_TOKEN" \
    -H "Content-Type: application/json" -H "Accept: application/json, text/event-stream" \
    "$@" --data "$body" "$URL" || true
}
header() {  # case-insensitive, any whitespace after the colon
  { tr -d '\r' < "$1" | grep -i "^$2:" | tail -1 \
    | sed -E 's/^[^:]*:[[:space:]]*//; s/[[:space:]]+$//'; } || true
}
is_json() { case "$(header "$1" content-type)" in application/json*) return 0;; *) return 1;; esac; }
stop() { echo "inconclusive: $*"; exit 2; }

code="$(post init '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-11-25","capabilities":{},"clientInfo":{"name":"session-probe","version":"0.1"}}}')"
is_json "$TMP/init.hdrs" || stop "initialize status=$code, response not JSON"
[ "$code" = 200 ] && jq -e '.id == 1 and (.result.protocolVersion | type == "string")' \
  "$TMP/init.body" >/dev/null 2>&1 || stop "no InitializeResult for id 1 (status=$code)"
PV="$(jq -r '.result.protocolVersion' "$TMP/init.body")"
[ "$PV" = "2025-11-25" ] || stop "negotiated $PV: these rules are for 2025-11-25"
SID="$(header "$TMP/init.hdrs" mcp-session-id)"
[ -n "$SID" ] || stop "no session ID observed: session comparisons do not apply"
FP="$(printf %s "$SID" | openssl dgst -sha256 -hmac "$FP_KEY" | awk '{print $NF}' | cut -c1-16)"
common=(-H "MCP-Protocol-Version: $PV" -H "MCP-Session-Id: $SID")
code="$(post initd '{"jsonrpc":"2.0","method":"notifications/initialized"}' "${common[@]}")"
[ "$code" = 202 ] || stop "notifications/initialized got status=$code, expected 202"

probe() {  # set LABEL first, for example LABEL=replica-a probe
  local code result=inconclusive
  code="$(post probe '{"jsonrpc":"2.0","id":2,"method":"ping"}' "${common[@]}")"
  if [ "$code" = 200 ] && is_json "$TMP/probe.hdrs" \
     && jq -e '.id == 2 and .result == {}' "$TMP/probe.body" >/dev/null 2>&1; then result=ok
  elif [ "$code" = 404 ]; then result=not_found; fi
  echo "$(date -u +%FT%TZ) label=$LABEL session_fp=$FP status=$code result=$result body=$(head -c 120 "$TMP/probe.body" | tr -d '\n')"
}
LABEL=baseline probe

A ping counts as ok only when it returns the empty result the ping specification requires. Anything else is marked inconclusive.

Two limits are deliberate. A server may answer a POST with an SSE stream, which can close before the response and be resumed later; this script does not handle that, so if your server streams, run the same comparisons with your SDK's client. And "no session ID observed" is not a diagnosis. Issuing a session ID is optional in 2025-11-25, but an intermediary that strips the header gives the same observation. Check the upstream's own response before calling the server sessionless.

ping invokes no application tool and writes nothing. It is not invisible to the session, though. The MCP Ruby SDK documentation says activity that touches a session, including a regular-request POST, resets its idle timer, and the ping specification says excessive pinging should be avoided. Probe at a bounded rate. When you test expiry, leave the target session untouched for the whole interval and probe once afterwards: a session pinged every minute tells you nothing about idle expiry. Use tools/list if you want the tool list, never a tool that writes. Our MCP tool authorization test matrix covers testing boundaries without crossing them.

The matrix: six comparisons, each changing one thing#

Each row changes one variable and keeps the rest fixed: same identity, same path, same protocol version, same read-only method. Every row assumes the probe's initialization gates passed; if they did not, the row is inconclusive.

Row 6 uses a fresh session on each route because a proxy that holds its own upstream session may have given your client an ID the upstream never saw. When both routes fail, correlate the session fingerprint, the receiving process and the request path for each route. If upstream logs confirm the upstream generated the response, record that as its own fact, separate from the diagnosis: where a 404 was generated is not who caused it.

Changing routing alone does not prove process-local state. If pinning to replica B also changes the path, the middleware or the build, a 404 there has more than one explanation. Row 2 supports replica-local state only when row 3 also shows the old session failing while a fresh one succeeds.

Do not create routes to run this. If no permitted pinning mechanism or upstream route exists, mark rows 2 and 6 inconclusive.

What the public reports show, and what they do not#

An mcp-framework issue describes the routing case. In version 0.2.22, the reporter's http-stream transport kept sessions in an in-memory map indexed by MCP-Session-Id. With two replicas behind a round-robin balancer, a request routed to the pod that did not create the session got 404 Session not found from that upstream: the server generated the 404, and routing caused it. The reporter adds that sessions are also lost when a pod restarts or is replaced in a rolling deployment. It is one account of one framework version.

The Ruby SDK documentation states the same constraint for its own transport: on the legacy handshake lifecycle it keeps session and SSE stream state in memory, must run in a single process, and needs sticky sessions keyed on Mcp-Session-Id across instances. It also shows the expiry case: stateful sessions expire after session_idle_timeout, 1,800 seconds by default. A client that sends nothing for 31 minutes against that default gets a 404 from the replica that issued its session. Affinity would not change that.

mcp-proxy issue 149, opened in December 2025, shows the client side. After a backend restart, the reporter saw the proxy keep sending the old session ID, get 404 and never reinitialize, so tool calls failed until someone restarted the proxy. The issue was closed on 2 November 2026 with a commit titled "recover lost upstream sessions without replaying operations". Treat it as history, not a current defect.

Its lesson is row 5, with a condition. Requests already in flight when the first 404 arrived come back 404 too, even from a client that then reinitializes correctly. Failed recovery shows when a hop sends new requests with the stale session after it received the 404, with no initialize in between.

A 2025-11-25 404 means reinitialize, not replay#

When a client receives HTTP 404 in response to a request containing an MCP-Session-Id, it "MUST start a new session by sending a new InitializeRequest without a session ID attached." Two limits matter.

It is a 404 rule. The issue 149 reporter proposed treating 401, 410, 503 and 500 the same way. The spec says this about 404 only. A 401 is about credentials, and a new session does not fix them. Do not carry the rule to other status codes or other revisions.

Starting a new session is not resending the request that failed. Re-establishing a session costs a handshake. Replaying the interrupted call can repeat a side effect. The spec has the server answer a terminated session with 404, which suggests the request was not processed there. Two gaps remain: the 404 may have come from a proxy, and an earlier call whose response never arrived has an unknown outcome. Before a client replays anything automatically, check whether the operation has side effects and whether the tool has an idempotency contract. Our article on tool timeouts, retries and idempotency covers that decision.

A new session also starts empty. An open GET stream or session-scoped state on the server has to be set up again.

Remedies match causes, and each has a cost#

Affinity fits process-local session state. It does nothing for expiry and does not survive a restart or a rollout: the replica the balancer remembers is gone, or is a new process with empty memory. If later routing uses only a newly issued random MCP-Session-Id, the balancer needs a way to associate that ID with the replica that issued it, because the initialize request carried none. Hashing that header alone does not guarantee the association. Affinity on another stable routing key is a separate design. That point is our reasoning, not a claim about any product.

Lifecycle-aware reinitialization is what the spec requires after a qualifying 404. It restores operation after an expiry or a finished restart, once the new initialize and the requests after it reach a working session. It does not repair the cause: with process-local sessions and non-affine routing, the new initialize can land on replica A and the next request on replica B, and the 404s repeat. It does not preserve session state, and it must not replay side effects.

Stateless mode, where an implementation offers it, removes session IDs at a cost. In the Ruby SDK's stateless mode, request-scoped notifications such as progress and log messages are skipped, and server-to-client requests (sampling, roots, elicitation) raise an error. Check your own SDK's documentation for its trade-offs.

Affinity plus client reinitialization covers more causes than either alone. Neither covers a server that loses sessions faster than clients use them.

2026-07-28 changes the question#

The 2026-07-28 transport overview describes a different lifecycle. Every request carries its protocol version and client capabilities in _meta fields in the body, and Streamable HTTP mirrors selected fields into HTTP headers so intermediaries can route without parsing the body. The overview says earlier revisions "established a connection-scoped session with an initialize handshake", and that clients and servers interoperating with them detect the counterpart's era and fall back.

So one deployment may serve both eras, and the 404 rule above belongs to only one. Record the negotiated version per client.

Write down what stays inconclusive#

The matrix narrows causes. It does not always name one. A restart that coincides with an idle gap, a proxy whose session state you cannot see, a server that answers in SSE: record these as inconclusive rather than picking the likeliest story, and add the missing timestamp or process identity to the next probe.

If your MCP traffic passes through Worksi's MCP gateway, it checks each tools/call against tool policy and records every call as a span, which adds one timestamped hop to the timeline. It does not document session affinity, automatic reinitialization or replay, so treat it as one more hop to account for, not as a fix for lost sessions.