Trace redaction protects one path: choose fields by question, then test every destination

Trace redaction is easy to set up as one switch: turn on masking, pick a capture level, move on. That treats an AI agent's telemetry as one pipe. A run can have several independent ways for sensitive data to leave your process: a framework's built-in exporter, your own OpenTelemetry exporter, application logs, baggage headers on downstream calls and the model request itself. Redaction configured on one of those paths tells you nothing about the others.

So this guide argues two things. First, decide what to capture from the questions you need to answer during an incident, not from what the SDK records by default. Second, check the result with synthetic canary strings, inspected at every destination separately, before any real user data flows.

Start from the diagnostic question, not the field#

A reasonable starting point is structure plus a small, reviewed set of metadata: which operation ran, in what order, how long it took, what error category it ended with, which model and code version were involved, and a correlation ID that lets you find the run again. That set can locate operational errors the trace recorded: a failed step, a timeout, a slow provider, a cost jump. It cannot tell you whether a call that succeeded did the right thing. For that you need content, or evidence from outside the trace.

Write the questions down first, then the minimum fields for each. A starting table:

The bottom four rows are the cost of minimization. Note what even full content gives you there: a record of what was sent and what was available, not how the model interpreted it. A captured passage shows the model had that text, not that the text caused the answer. To argue about interpretation you compare the next output or action against what a correct reading would have produced, and the conclusion is still an inference.

A tool span that completed with a clean status says the call happened, not that the outcome was right. We wrote a checklist for green tool spans with wrong outcomes, and several of its checks need captured inputs and outputs plus a record from the source system. With metadata only, those questions move out of the trace and into a reproduction, or into the system of record.

That trade can be narrowed. Langfuse's guide to PII masking patterns recommends redacting known sensitive fields by name or path rather than blanking whole texts, because field-level redaction keeps the structure readable. If "what order id did the tool receive" matters to you, capturing that one argument may be a better answer than capturing the whole prompt.

Metadata is smaller, not safe#

Fewer fields reduce what can leak. They do not make what remains anonymous. Treat each metadata field as something to review, not something exempt from review:

• Names can carry user data. Agent, workflow, tool and span names are strings your code builds. A hypothetical example: a span named summarize_ticket_for_jane because a developer interpolated the customer's name. A redactor that only inspects payload fields, or only matches known patterns, can leave that name untouched. Some redactors do cover names: the OpenAI Agents SDK's tracing docs show a redaction example that omits names through an allowlist. Check what yours inspects.
• Error messages carry content. A validation error that echoes the bad value puts tool arguments in a field labeled "error".
• Hashes of predictable inputs are reversible. OpenTelemetry's sensitive data guidance shows collector configs that hash user.email or replace user.id with a SHA-256 hash, and then states the limit: hashes "are reversible in practice if the input space is small and predictable (e.g. numeric user IDs)". The same reasoning applies to an argument fingerprint over a short order number.
• Correlation IDs need the same care. The same Agents SDK page suggests a chat thread ID as a group_id, and says of its redaction example that caller-supplied IDs must contain no sensitive information, or the application must map them to safe values. A random per-run id that you map to a user in your own database is safer to export than the user id itself.

The OpenTelemetry page also lists the minimization rules that make this table useful: collect only what serves an observability purpose, and regularly review collected attributes to ensure they remain necessary.

Every path is its own boundary#

Draw the paths before configuring anything. For an agent built on a framework with its own exporter:

1. In-process instrumentation. The framework and your own code create spans and attributes.
2. Each exporter. A framework's default exporter, your OTLP exporter and any vendor SDK each serialize and send on their own schedule.
3. The collector, if one exists. It can delete, hash or filter attributes before forwarding.
4. Each backend. Whatever stores and indexes the data.
5. Logs. Application logs, exception trackers and debug prints that sit beside tracing.
6. Propagated context. OpenTelemetry's baggage docs warn that automatic instrumentation includes baggage in most of a service's network requests, in HTTP headers, so sensitive baggage items can reach third-party APIs.
7. The provider request. The prompt your model provider receives.

Two consequences follow. A collector processor that deletes an attribute acts on data that has already left the application and crossed the network to reach it. It cannot undo that transmission, and it does not see an exporter that sends straight to a vendor. And the provider request is a separate boundary altogether. The Langfuse guide puts it plainly: masking for observability does not change what the model provider sees. A prompt can carry data the model needs while the stored trace redacts it, or the trace can keep content the provider never received. If certain data must not reach the provider, the control has to sit on the request path, and it has to be tested there.

Pattern-based redaction has a second limit that no placement fixes. The Langfuse guide calls regex scrubbing a coarse safety net and says to expect false negatives on free text. An email address has a shape. "Please ship it to the woman at the corner house on Mill Road" does not.

What the OpenAI Agents SDK sends by default#

The Agents SDK is a useful example because its docs are specific. Read on 9 November 2026, the tracing page says:

• Tracing is on by default. The default setup is a BatchTraceProcessor that sends spans to a BackendSpanExporter, which exports to OpenAI's backend.
• generation_span() stores the inputs and outputs of the LLM generation, and function_span() stores the inputs and outputs of function calls. RunConfig.trace_include_sensitive_data controls that capture, and it defaults to True. The OPENAI_AGENTS_TRACE_INCLUDE_SENSITIVE_DATA environment variable sets the default without code.
• With the setting False, Responses model spans omit request input and response output, but for official OpenAI endpoints still include the response_id as correlation metadata.
• add_trace_processor() adds a processor and leaves the default OpenAI exporter registered. set_trace_processors() replaces the defaults.
• Processors are independent observers. The provider catches a processor's callback exceptions and keeps calling the others, so a redaction processor registered before an exporter does not stop that exporter receiving data if redaction fails.

That last point matters most here. "I added a redaction processor" and "nothing unredacted leaves" are different statements. The docs' recommended composition, when export depends on successful redaction: replace the default processors with set_trace_processors(), use a BatchTraceProcessor with an exporter you own, have that exporter copy the payloads, redact the copies and send only the redacted results, and drop the batch if serialization, copying or redaction fails. That fail-closed behavior is something your exporter implements; the docs describe it as the application's job, not as what the default exporter does. They also note that replacement does not erase data already buffered by a previously registered processor, so configure it before creating traces or running agents.

This describes the documentation on one date, for the Python SDK. Defaults move between versions. The canary test below is how you find out what your installed version does with your configuration.

Run a canary test across every destination#

A canary is a unique synthetic string you plant in one place and then search for everywhere. Give each placement its own string, so a hit tells you which field reached which destination.

import secrets

RUN = secrets.token_hex(4)
PLACES = ["prompt", "tool-arg", "attr", "span-name", "error", "baggage"]
CANARIES = {p: f"cnry-{p}-{RUN}-{secrets.token_hex(6)}" for p in PLACES}

# Synthetic free-text personal data with no pattern to match:
FREE_TEXT = f"Please send it to Imogen Vantrell at 14 Quillfeather Lane ({RUN})"

Plant each value in a test environment that uses the same configuration as production:

• prompt: inside the user message, together with FREE_TEXT.
• tool-arg: as an argument the model will pass to a tool. A tool that echoes an instruction field works.
• attr: as a custom span attribute.
• span-name: in a custom span or workflow name.
• error: raised in a tool exception message.
• baggage: as a baggage entry before the run starts.

Then search every destination for RUN and record each result in a matrix:

Fill in each cell with what you expect before you run the test, then compare. A hypothetical filled-in error row, for a setup that logs tool exceptions and runs a redacting exporter in front of the backend, might read:

Here the backend cell is the finding. The collector received the redacted value, so something between the collector and the backend, or a second path into the backend, carried the raw exception text. The logs hit is expected, and it is still a destination holding the canary.

Some rules that make the result trustworthy:

• Inspect each path at its own exit. Check the collector's input as well as the backend, or you cannot tell "the collector removed it" from "it never reached the collector".
• Check the provider request separately. You need a view of what was sent upstream in the test environment: a gateway's request record or a local proxy. Trace storage tells you nothing here.
• Expect pattern redaction to miss the free-text row. Generic pattern redaction may leave the synthetic name and street intact. If they come through, compare the result with the policy you declared for that field and confirm the redactor ran at all: the same leak appears when a redactor is bypassed or broken. Then choose the control: omit the field, redact it by structure, use a stricter capture level, or another control you evaluate the same way.
• Rerun after upgrades. SDK, instrumentation and collector versions change what gets recorded. The OpenTelemetry guidance makes reviewing the telemetry of every instrumentation library you use your responsibility.
• Keep canaries synthetic. The point is to learn where data goes without moving real data there.

An empty cell where you expected a hit needs the same attention as an unexpected hit. If the prompt canary is missing from a backend that should hold full content, the expected path failed somewhere, and you do not yet know where. Confirm the test reached the model call. Wait for the exporter's batch, or flush it explicitly: the Agents SDK's default batch processor exports in the background. If you implemented the application-owned exporter described above, check whether it discarded the batch after serialization, copying or redaction failed. Then check capture settings, filtering or sampling, exporter delivery and backend ingestion one at a time before you name the cause.

A passing matrix verifies the inputs you planted, on the paths you inspected, with the versions you ran. It says nothing about a field or a destination you left out.

Where Worksi fits, and where it does not#

Worksi stores traces under a capture mode set per environment: Full, Redacted (the default, which replaces sensitive values) or Metadata only (hashes, counts, tokens, cost, timings, status and error types, with no previews). A client can make it stricter per request, for example with an X-AIW-Capture: metadata_only header, never looser. Redaction is pattern-based: it does not detect names, postal addresses or free-text personal data, and agent, workflow and tool names, tags and attributes are stored as sent.

Capture mode governs what Worksi stores. It is not a control on your other exporters, such as a framework's default trace exporter, and it does not change what your provider receives through the gateway. The canary matrix still applies, with Worksi as one more column.