AI Observability: What to Instrument, What It Costs, What It Misses

What AI observability actually means

Every vendor definition of AI observability says roughly the same thing: collect telemetry across models, prompts, retrieval and agent steps so you can understand behavior in production. That is correct and not very useful, because it describes the collection and skips the hard part. AI systems need a separate practice not because they produce more telemetry, but because the thing you care about, whether the output was any good, appears in no conventional signal.

A classical service tells you it failed. It returns a 500, a timeout, a saturated queue. A language model almost never fails that way. It returns a well-formed answer in eight hundred milliseconds, and the answer cites a policy that does not exist. Latency, error rate and throughput all look fine. The dashboard is green and the product is broken.

So AI observability is two separable jobs. The first is reconstruction: given a request ID or a complaint, can you replay what happened, which model version, which system prompt, which retrieved chunks, which tool calls with which arguments, at what cost. The second is judgment: on what basis do you decide the output was wrong. The first is engineering you already know. The second requires you to define quality for your own domain, and no platform can do that for you. Sources: OpenTelemetry.

How it differs from the monitoring you already run

Written side by side, the gap is obvious. Most teams already own the left column and have nothing in the right one.

What to instrument, concretely

The OpenTelemetry GenAI conventions are the most useful public answer to this question, because they were negotiated across competing vendors rather than written to sell one product. They define four kinds of GenAI span: inference, when a client calls a model; embeddings; retrievals, for vector store searches; and memory operations. Two attributes are required on all of them, the operation name and the provider name, and the recommended set covers the request model, sampling parameters, finish reasons, and token usage split into input, output, cache creation and cache read. Sources: OpenTelemetry.

That last split matters more than it looks. Once prompt caching is in play, a single aggregate token counter quietly misreports spend, because cache reads and cache writes are priced differently from fresh input. If your cost dashboard has one number called tokens, it is already wrong. Sources: OpenTelemetry.

Underneath the GenAI layer, nothing exotic is required. Trace context propagation across service boundaries is a W3C Recommendation from 2021, with traceparent carrying the position of a request in its trace graph and tracestate carrying vendor data. If your services already propagate those headers, an AI call is just another span in a trace you already have, and the work is instrumenting the model client rather than rebuilding a tracing backbone. Sources: W3C.

The standard is real, and it is not finished

This is the part the buyer's guides leave out. The GenAI semantic conventions now live in their own OpenTelemetry repository, separate from the core conventions, covering spans, metrics, events, MCP, and provider-specific pages. The span document carries the status Development at the top, and at the time of writing the dedicated repository has published no tagged releases at all. Sources: OpenTelemetry; OpenTelemetry.

Development status has a precise practical meaning: attribute names can change, and code you write against them today may need updating. That is not a reason to wait. It is a reason to put your instrumentation behind a thin layer of your own so a rename is a one-file change, and to prefer backends that treat the convention as an input rather than a private format. Langfuse, for example, states that it aims to be compliant with the OpenTelemetry GenAI semantic conventions while noting those conventions are still evolving, and maps incoming spans into its own model. MLflow describes its tracing as fully compatible with OpenTelemetry with native support for the GenAI conventions on both export and ingestion. Arize Phoenix accepts OTLP rather than a proprietary wire format. Sources: Langfuse; MLflow; Arize Phoenix.

The OpenTelemetry project is explicit about why this work exists at all: agent applications need standard telemetry shapes to avoid lock-in, and the same telemetry doubles as a feedback loop for improving agent quality. Treat that as the selection test. A platform that can only be fed by its own SDK is a platform you cannot leave. Sources: OpenTelemetry.

Observability without evaluation is a very expensive log

Tracing tells you what happened, not whether it was correct. The bridge is evaluation, and the pattern is circular: production traces become evaluation datasets, evaluation produces scores, scores attach back to traces, and the aggregate becomes the quality metric you were missing. MLflow describes this loop, capturing inputs, outputs and metadata for each intermediate step and using the result for quality evaluation and dataset creation from real traffic. Sources: MLflow.

For scoring at volume, model-based judging is the common answer. Google's Gen AI evaluation service supports model-based metrics where models assess outputs against specified criteria, alongside custom metrics and dedicated agent evaluation. OpenAI's evals guide frames the same idea as testing outputs against criteria you specify, with graders deciding whether an output is correct, run over datasets of test items with ground-truth labels. Sources: Google Cloud; OpenAI.

Two cautions, both from the primary sources rather than from opinion. First, a model-based judge is itself a model, subject to the same drift and the same unverified confidence as the system it grades, so a judge needs its own agreement check against human labels before you trust its trend line. Second, this tooling churns. OpenAI's own evals documentation currently states that the Evals platform is being deprecated and is scheduled to shut down on 30 November 2026. If your quality history lives only inside one vendor's eval product, your quality history is on a clock. Sources: OpenAI.

Cost and privacy are the two limits people hit first

AI traces are fat. A classical span is a few hundred bytes of metadata. A GenAI span can carry a system prompt, a full conversation, several retrieved documents and a long response. Teams that turn on full payload capture across all traffic tend to find that observability has become a line item next to inference itself.

OpenTelemetry's guidance on sampling is directly applicable and worth reading before you buy anything. It calls sampling one of the most effective ways to reduce observability cost without losing visibility, and notes that for high-volume systems a rate of one percent or lower can accurately represent the rest. It is equally frank about the trade-offs: head sampling is simple but cannot decide based on the whole trace, so you cannot guarantee every error trace is kept, while tail sampling buys that guarantee with stateful infrastructure, operational complexity, and a lock-in risk where the best options are limited to what your vendor offers. Sources: OpenTelemetry.

The workable compromise is asymmetric: sample aggressively on the metadata path, keep every trace carrying an error, a low quality score, a safety flag or a complaint, and keep full prompt payloads for a short window only. Which leads to the second limit.

Recording prompts and responses means recording whatever your users typed, which routinely includes personal data, credentials pasted by mistake, and customer records pulled in by retrieval. That content now sits in a third-party analytics system with its own retention policy and access model. Redact at the instrumentation layer rather than at the backend, set a retention window you can defend, and check the provider's data terms: Anthropic, for instance, documents how zero data retention applies to individual API features. Treat a trace store holding raw prompts as a production data store, because that is what it is. Sources: Anthropic.

One more cost trap that observability is supposed to catch and usually does not. Token accounting is not stable across model generations. Anthropic's documentation states that Claude 4.7 and later models use a newer tokenizer, that the same input text produces approximately thirty percent more tokens than on earlier models, and that you should recount prompts against the model you plan to use rather than reusing earlier counts. A cost-per-request chart that spans a model migration is therefore comparing two different units. Version your cost baselines by model, or your capacity planning will silently drift. Sources: Anthropic.

Agents make all of this harder

A single model call is one span with a clear beginning and end. An agent is a loop with a variable number of iterations, tool calls that mutate external state, memory that persists between runs, and, increasingly, remote tool servers the agent reaches over a protocol. The same input can take a different path on Tuesday than it took on Monday, which breaks the reproduction step that debugging depends on. Sources: OpenTelemetry.

Three failure modes matter more than raw latency, and each has a specific telemetry answer. Silent looping, retrying a failing tool until the budget is gone, needs a per-run iteration count and a hard ceiling. Wrong action, calling the right tool with wrong arguments, needs tool arguments and results as structured span attributes rather than free text. Scope creep, using a capability nobody expected, needs the permission set recorded at run time.

Two of these map to named risks in the OWASP Top 10 for LLM Applications 2025, which lists Excessive Agency at LLM06 and Unbounded Consumption at LLM10. That is a useful framing for an engineering argument, because it turns an observability request into a security requirement with a reference behind it, which tends to survive prioritization meetings better than a request for better dashboards. Sources: OWASP.

• Record the iteration count and stop reason on every agent run, not only the final answer.
• Capture tool name, arguments, result status and duration as attributes, redacting payloads.
• Keep the resolved model version, system prompt version and tool schema version on the root span.
• Log the permission set the run actually had, so scope changes are visible after the fact.
• Keep every trace that ended in an error, a safety flag, a low score, or an escalation.

What regulators already expect you to keep

There is a compliance argument for this work, and it is stronger than most teams realize. NIST's AI Risk Management Framework 1.0, published on 26 January 2023, is organized around four functions, Govern, Map, Measure and Manage. Measure and Manage are not satisfiable without production telemetry, because you cannot measure or manage a deployed system whose behavior you do not record. Sources: NIST.

In the European Union the requirement is explicit rather than implied. Article 12 of Regulation (EU) 2024/1689 states that high-risk AI systems shall technically allow for the automatic recording of events, logs, over the lifetime of the system, so that risk situations can be identified, post-market monitoring is possible, and deployers can monitor operation. The Commission's own timeline puts general application at 2 August 2026, two years after entry into force, with exceptions: prohibitions applied from February 2025, governance and general-purpose model obligations from August 2025, and obligations for several high-risk categories arriving later, from December 2027 and August 2028. Sources: AI Act Explorer; European Commission.

The practical reading for a team outside the high-risk categories: you are probably not obligated to keep these logs, and you will be asked for them anyway, by a customer security review, an incident postmortem, or a procurement questionnaire. Building the capability early is cheaper than retrofitting it under a deadline.

A rollout order that survives contact with production

Most failed observability projects fail the same way: buy a platform, turn on full capture, generate a large bill and a pile of traces nobody reads, conclude the category is overhyped. The order below inverts that, and each step is worth doing on its own.

• Start with one trace per request carrying model version, prompt version and token usage. That alone answers most cost and regression questions.
• Add tool and retrieval spans next, where agent failures actually live.
• Define one quality metric for one user-facing task, in writing, before evaluating anything.
• Label a few hundred real production traces by hand. Unglamorous, and the step that makes every later number meaningful.
• Only then add an automated judge, and validate it against those human labels before trusting its trend.
• Set retention and redaction on the day you turn capture on, not after the privacy review.
• Keep the emission layer thin and standards-shaped, so changing backends is a config change.

What the Roundup's own coverage suggests

Reading the daily AI industry signals we track, this is not really a monitoring story. It is a consequence of two other shifts. Agent frameworks moved from demos into production workflows, turning a one-shot API call into a long-running process with side effects. And the infrastructure layer consolidated, making model portability a commercial question rather than a technical one. Observability is where those pressures meet: the layer that tells you what your agents did, and the layer that decides whether you could move them somewhere else.

The same argument applies to the components underneath the agent, which we worked through in our guide to open-source AI agents: the useful question is never whether a thing is open or observable in the abstract, but which specific layer you would still control if you changed your mind next quarter.

Limits and uncertainty

This guide describes a practice and a standards landscape, not a benchmark, and it deliberately avoids ranking tools. Three limits are worth stating plainly. The OpenTelemetry GenAI conventions are in Development status with no tagged release in their dedicated repository, so specific attribute names here may change, and the durable advice is the shape of the telemetry rather than the exact keys. The product examples show how each project describes its own OpenTelemetry compatibility, not an endorsement or a comparison, and that documentation is continuously updated, so it was read on 27 July 2026. And observability has a hard ceiling: it can tell you what a system did and what it cost, but whether an output was correct always rests on a definition of quality your team writes and maintains. A quality score nobody has validated against human judgment is a number, not a measurement.