RAG and agent observability: scoring retrieval and generation separately, then the trajectory

Three RAG failure modes and their detection signals

Every RAG failure traces back to one of three root causes. Naming them precisely is the first step toward instrumenting them.

RAGAS: the four core metrics

Es et al. formalised the evaluation framework in “RAGAS: Automated Evaluation of Retrieval Augmented Generation” (EACL 2024) [2]. RAGAS defines four metrics that span both retrieval and generation quality, each measurable without human annotation for every query:

• Faithfulness — fraction of answer claims supported by the retrieved context. Low faithfulness with high context precision = context-integration failure.
• Answer relevance — how directly the generated answer addresses the question. Measures whether the model stayed on topic.
• Context precision — proportion of retrieved chunks that are relevant to the question. Low context precision = noisy retrieval.
• Context recall — whether all necessary evidence was present in the retrieved set. Low recall = retrieval failure (coverage gap).

The TruLens projectframes the same decomposition as the “RAG Triad”: context relevance, groundedness (equivalent to faithfulness), and answer relevance [3]. The naming differs; the decomposition is the same. Whether you adopt RAGAS, TruLens, or a comparable framework (e.g. DeepEval), the important design decision is that you instrument all three dimensions — treating them as separate signals in your observability pipeline rather than averaging them into a single score.

OpenTelemetry GenAI semantic conventions: the instrumentation vocabulary

The OpenTelemetry semantic conventions for generative AI systems [4] define a standardised attribute namespace — gen_ai.* — that lets you capture LLM and agent signals in a vendor-neutral way across any OTel-compatible collector. This is the layer that converts per-vendor SDKs into a common observability substrate.

Key attributes from the GenAI namespace that every RAG and agent pipeline should emit:

• gen_ai.system — identifies the model provider (e.g. openai, anthropic, vertex_ai). Enables cross-provider comparison in dashboards.
• gen_ai.request.model / gen_ai.response.model — the model name and version requested. Pairs to detect model routing or fallback events.
• gen_ai.usage.input_tokens / gen_ai.usage.output_tokens — token counts per call. Essential for cost attribution per RAG query or agent run.
• gen_ai.operation.name — the operation type (chat, text_completion, embeddings). Lets you separate embedding calls (retrieval) from generation calls on the same span trace.
• gen_ai.tool.name / gen_ai.tool.call.id — the name and call identifier of a tool invoked by the model. The primary grading unit for agent trajectory evaluation.
• gen_ai.agent.id / gen_ai.agent.name — defined in the agent spans specification [4b], these identify the agent instance so you can correlate tool calls back to a named agent across a multi-agent trace.

Concretely, a RAG query should emit at minimum two child spans: one for the embedding call (operation.name = embeddings, carrying the query text and token count) and one for the generation call (operation.name = chat, carrying the model, token counts, and the retrieved-context hash or identifier). A span hierarchy of this shape, collected by any OTel-compatible pipeline and stored in a traces backend (e.g. Tempo, Jaeger, or a managed provider), gives you the raw material to compute all four RAGAS dimensions without instrumenting proprietary SDK events.

Agent trajectory evaluation: grading intermediate steps

An agent system executes a plan across multiple steps: it calls tools, processes results, updates its state, and eventually produces a final output. End-to-end outcome evaluation (“did it produce the right answer?”) misses a class of reliability problems that only surface in the trajectory — the sequence of states, tool calls, and reasoning steps the agent took to get there.

What trajectory grading measures

A well-instrumented trajectory evaluation grades three properties independently:

1. 1Tool-use correctness — did the agent call the right tools, with the right arguments, in the right order? A trajectory that calls a search tool twice when once is sufficient, or calls a write tool before reading the current state, is wasteful or unsafe even if the final output is correct.
2. 2Plan validity — does the sequence of steps reflect a coherent plan for the stated goal? This catches agents that succeed by brute-force retrying rather than by reasoning correctly, and agents that abandon a valid plan mid-execution.
3. 3Intermediate-step coherence — at each step, is the model's reasoning consistent with the evidence it has accumulated? This is the per-step analogue of faithfulness: the model should not assert facts at step N that contradict the tool output it received at step N-1.

Three evaluation frameworks and what each grades

Connecting evaluation to the observability pipeline

Evaluation scores are only operationally useful if they flow into the same observability pipeline as infrastructure metrics. The pattern is to emit RAGAS and trajectory scores as OTel metrics (using a counter or histogram instrument) tagged with the pipeline name, model version, and query category. This makes them queryable alongside latency and token-cost metrics in the same dashboards and alerting rules — so you can correlate, for example, a spike in context-integration failures with a model rollout, or a drop in context precision with an index rebuild.

A minimal RAG observability instrumentation checklist:

# Minimum OTel attributes per RAG query span
embedding_span:
  attributes:
    - gen_ai.system          # e.g. "openai"
    - gen_ai.operation.name  # "embeddings"
    - gen_ai.request.model   # e.g. "text-embedding-3-small"
    - gen_ai.usage.input_tokens

generation_span:
  attributes:
    - gen_ai.system
    - gen_ai.operation.name  # "chat"
    - gen_ai.request.model
    - gen_ai.response.model  # detect routing/fallback
    - gen_ai.usage.input_tokens
    - gen_ai.usage.output_tokens

# Custom evaluation metrics (emit as OTel metrics, not spans)
evaluation_metrics:
  - rag.faithfulness          # float 0-1
  - rag.context_precision     # float 0-1
  - rag.context_recall        # float 0-1
  - rag.answer_relevance      # float 0-1
  - agent.tool_call_correctness   # float 0-1, per agent run
  - agent.step_coherence          # float 0-1, per step

Storing these as OTel metrics with a Prometheus-compatible scrape endpoint means you can write alerting rulesof the form “alert when rag.faithfulness p50 drops below 0.7 over a 1-hour window” — the same pattern you would use for API error-rate alerts. Evaluation quality becomes a first-class operational signal, not an offline analysis step.

Separating the failure modes in practice

A useful diagnostic pattern when a RAG system degrades is to compute all four RAGAS metrics on the failing queries simultaneously and read the pattern:

• Low context precision + low context recall, high faithfulness → retrieval failure. The model is faithfully using what it was given; it was just given the wrong material. Fix: embedding model, chunking strategy, or query expansion.
• High context precision + high context recall, low faithfulness → context-integration failure. Good context was retrieved; the model ignored it. Fix: prompt engineering, reducing context window noise, or model choice.
• All metrics stable, but answer correctness drifting on a golden test set → corpus freshness failure. Re-index the corpus and update staleness monitoring.
• Low answer relevance with high faithfulness and precision → the model is staying grounded but answering a different question than the one asked. Usually a query understanding or prompt-framing issue.

What this means for platform design

For platform engineers supporting RAG and agent workloads, the practical implications are:

1. 1Require OTel GenAI instrumentation at the workload level. RAG pipelines and agent frameworks that do not emit gen_ai.* spans cannot be observed. Make it a platform onboarding requirement, not an optional add-on.
2. 2Provide evaluation pipelines as platform primitives. Running RAGAS or Inspect on a scheduled basis against a golden test set should be a platform-managed job — similar to how integration tests run in CI — not something each application team re-implements independently.
3. 3Store trajectory data for offline analysis. Agent span traces are large; store the full trace for a sample fraction (e.g. 10%) and aggregate metrics for the rest. The full traces are needed when you need to debug a failure mode that only appears in multi-step sequences.
4. 4Treat corpus freshness as an infrastructure metric, not an application concern. Index age, re-indexing job success rate, and document-staleness histograms belong in the same SLO framework as database replication lag.