Exported spans and diagnostic events

Exported spans

• openclaw.gateway.rpc.response, openclaw.gateway.rpc.handler, openclaw.gateway.rpc.dispatch

  • Completed phase observations with openclaw.gateway.rpc.method, openclaw.gateway.rpc.phase, and openclaw.gateway.rpc.outcome
  • Handler spans include openclaw.gateway.rpc.admission_ms; dispatch spans include openclaw.gateway.rpc.response, the response state at dispatch settlement
  • Preserve a supplied upstream request parent; they do not introduce a long-lived RPC parent span or change downstream trace propagation

• openclaw.model.usage

  • openclaw.channel, openclaw.provider, openclaw.model, optional openclaw.agent (the agent that owns the run when known)
  • Optional host-derived openclaw.plugin only for trusted plugin runtime completions
  • openclaw.tokens.* (input/output/cache_read/cache_write/total)
  • gen_ai.system by default, or gen_ai.provider.name when the latest GenAI semantic conventions are opted in
  • gen_ai.request.model, gen_ai.operation.name, gen_ai.usage.*

Plugin attribution is span-only. It does not add a plugin dimension to shared OpenTelemetry metrics or change Prometheus metric labels.

Embedded and simple model requests compute prompt component sizes only when a diagnostic subscriber requests them. Activity tracking, the stability recorder, Prometheus, and metrics-only OpenTelemetry do not request these sizes. Tracing retains the same pre-provider JSON character counts; these are distinct from the encoded request's UTF-8 byte count. Internal diagnostic subscribers can set includePromptStats: false in their event filter to avoid requesting this work. Omitting the option preserves prompt statistics for interested subscribers.

• openclaw.run
  • openclaw.outcome, openclaw.channel, openclaw.provider, openclaw.model, openclaw.errorCategory, optional openclaw.agent
• openclaw.model.call
  • gen_ai.system by default, or gen_ai.provider.name when the latest GenAI semantic conventions are opted in
  • gen_ai.request.model, gen_ai.operation.name, openclaw.provider, openclaw.model, openclaw.api, openclaw.transport, openclaw.model_call.observation_unit (request or turn), optional openclaw.agent
  • gen_ai.response.model on terminal request spans (completed and error) when the provider reported the model that actually served the request — e.g. routed OpenRouter presets, openrouter/auto, or alias models. gen_ai.request.model and openclaw.model keep the requested id; the response model is observed from the terminal message, omitted when absent, and never attached to aggregate usage or turn spans, used for OpenClaw pricing, or exported as a metric label (unbounded cardinality). model.call.started is emitted before the provider responds and never carries it.
  • openclaw.errorCategory, error.type, and optional openclaw.failureKind on errors
  • openclaw.model_call.request_bytes, openclaw.model_call.response_bytes, openclaw.model_call.time_to_first_byte_ms
  • openclaw.model_call.prompt.input_messages_count, openclaw.model_call.prompt.input_messages_chars, openclaw.model_call.prompt.system_prompt_chars, openclaw.model_call.prompt.tool_definitions_count, openclaw.model_call.prompt.tool_definitions_chars, openclaw.model_call.prompt.total_chars (safe component sizes only, no prompt text)
  • openclaw.model_call.usage.* and gen_ai.usage.* when the result carries usage for that request or aggregate turn
  • Cache-write totals remain in openclaw.model_call.usage.cache_creation_input_tokens and gen_ai.usage.cache_creation.input_tokens. When reported, openclaw.model_call.usage.cache_creation_input_tokens_1h and openclaw.model_call.usage.cache_creation_input_tokens_5m carry the provider’s TTL buckets verbatim, including zero; unreported buckets stay absent. TTL subsets are OpenClaw-specific so generic usage consumers do not price both the total and its breakdown.
  • Span event openclaw.provider.request with attribute openclaw.upstreamRequestIdHash (bounded, hash-based) when the upstream provider result exposes a request id; raw ids are never exported
  • Codex turn span event openclaw.model.response.observed with openclaw.upstreamRequestIdHash, openclaw.model_response.timestamp_source = receipt, openclaw.model_response.usage_source = native-normalized, openclaw.model_response.usage_scope = response, and known openclaw.model_call.usage.* / gen_ai.usage.* fields. It records an untimed native response observation and does not increment usage metrics.
  • With OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental, request spans use the latest GenAI inference span name {gen_ai.operation.name} {gen_ai.request.model}. Turn spans use invoke_agent because OpenClaw does not claim a native agent name from the opaque CLI boundary. Both use CLIENT span kind instead of openclaw.model.call.
• openclaw.harness.run
  • openclaw.harness.id, openclaw.harness.plugin, openclaw.outcome, openclaw.provider, openclaw.model, openclaw.channel, optional openclaw.agent
  • On completion: openclaw.harness.result_classification, openclaw.harness.yield_detected, openclaw.harness.items.started, openclaw.harness.items.completed, openclaw.harness.items.active
  • On error: openclaw.harness.phase, openclaw.errorCategory, optional openclaw.harness.cleanup_failed
  • Span event openclaw.agent.commentary for completed preambles from supported harnesses, including the built-in runtime, Codex, and Claude CLI. Attributes include openclaw.commentary.sequence, openclaw.commentary.text_length, and openclaw.commentary.content_truncated. The existing diagnostics.otel.captureContent setting controls bounded, redacted output-message content.
  • With captureContent: true: bounded, redacted input.value on start and output.value (the runtime's final message text) on completion. For the Claude CLI backend, input.value is the exact composed prompt the CLI received, after history reseeding, plugin input text transforms, and prompt context; the Codex harness records the turn prompt OpenClaw handed it. The Gemini CLI backend publishes captured content on openclaw.message.processed (inbound message and the model's final response); it does not record its composed CLI prompt on these run-level spans.
• openclaw.tool.execution
  • gen_ai.tool.name, gen_ai.operation.name (execute_tool), openclaw.toolName, openclaw.tool.source, optional gen_ai.tool.call.id, openclaw.tool.owner, openclaw.tool.params.*, optional openclaw.agent
  • Optional openclaw.errorCategory/openclaw.errorCode on errors, openclaw.deniedReason and openclaw.outcome=blocked when denied by policy or sandbox
• openclaw.skill.used
  • openclaw.skill.name, openclaw.skill.source, openclaw.skill.activation, optional openclaw.agent and openclaw.toolName
  • Optional openclaw.skill.fingerprint: SHA-256 of the complete instruction body actually delivered by read or skills_read. Cached reads reuse the delivered identity; unread support files and discovery results do not contribute. The fingerprint is omitted when no traced instruction delivery is known and never labels metrics.
• openclaw.exec
  • openclaw.exec.target, openclaw.exec.mode, openclaw.outcome, openclaw.failureKind, openclaw.exec.command_length, openclaw.exec.exit_code, openclaw.exec.exit_signal, openclaw.exec.timed_out
• openclaw.webhook.processed
  • openclaw.channel, openclaw.webhook
• openclaw.webhook.error
  • openclaw.channel, openclaw.webhook, openclaw.error
• openclaw.message.processed
  • openclaw.channel, openclaw.outcome, openclaw.reason, optional openclaw.agent (the agent that initially ingested the prompt)
  • Isolated cron agent turns use this span as the parent of their harness spans, keeping model calls, tools, and usage on the same trace through completion or failure.
  • With captureContent: true: bounded, redacted input.value (inbound message text) and output.value (the model's own response text; hook replies and host notices are never captured)
• openclaw.message.delivery
  • openclaw.channel, openclaw.delivery.kind, openclaw.outcome, openclaw.errorCategory, openclaw.delivery.result_count
• openclaw.session.stuck
  • openclaw.state, openclaw.ageMs, openclaw.queueDepth
• openclaw.context.assembled
  • openclaw.prompt.size, openclaw.history.size, openclaw.context.tokens, openclaw.errorCategory (no prompt, history, response, or session-key content)
• openclaw.tool.loop
  • openclaw.toolName, openclaw.loop.level, openclaw.loop.action, openclaw.loop.detector, openclaw.loop.count, optional openclaw.loop.paired_tool, optional openclaw.agent (no loop messages, params, or tool output)
• openclaw.memory.pressure
  • openclaw.memory.level, openclaw.memory.reason, openclaw.memory.rss_bytes, openclaw.memory.heap_used_bytes, openclaw.memory.heap_total_bytes, openclaw.memory.external_bytes, openclaw.memory.array_buffers_bytes, optional openclaw.memory.threshold_bytes/openclaw.memory.rss_growth_bytes/openclaw.memory.window_ms

When content capture is explicitly enabled, model and tool spans can also include bounded, redacted openclaw.content.* attributes for the specific content classes you opted into.

openclaw.message.processed and openclaw.harness.run spans follow the same existing captureContent setting. When it is true, input.value and output.value carry the inbound message or turn prompt and the model's final response text, bounded to 128 KiB (131072 UTF-16 code units) per field and redacted through the same sensitive-text filter as other exported content. output.value records what the LLM actually responded with — the model's own final response, before delivery filtering — so sentinel replies such as NO_REPLY appear verbatim; this is LLM-observability data, not a delivery record. Only the text of the model's last message counts. Text the model passed to the message tool is an argument to a tool call, not its final message, so tool-only turns record the model's closing message (usually NO_REPLY) or nothing; the sent text is on the child openclaw.model.call output messages and, for the built-in runtime, the openclaw.tool.execution span (gen_ai.tool.call.arguments). CLI backends do not export tool arguments, so their message-tool text is not captured on any span. Earlier narration in the turn ("Checking now…") is never reported as the final response: a message that ended in a tool call is not final, and CLI turns record only the final message, never the cumulative reply text. Text the model did not write is never captured, even when it was delivered to the chat: plugin hook replies (before_dispatch, before_agent_reply), no-visible-reply fallbacks, continuation statuses, failure texts, and Gateway restart notices. Turns where no model produced text carry no output.value. The captured content is routed as private listener-only payload data to trusted diagnostics exporters only; it never appears on public event payloads.

Upgrade note: if you already run with diagnostics.otel.captureContent: true, these message and run span attributes are new — your existing setting now also exports this prompt and reply content with no further configuration change. Nothing new is exported while captureContent is unset or false.

Diagnostic event catalog

The events below back the metrics and spans above. Public events are also available for direct plugin subscription; trusted core events such as model.usage are restricted to authorized internal consumers. run.progress and run.execution_phase are direct-only lifecycle signals; the diagnostics-otel plugin does not export them as standalone OTLP signals. Event kinds and run.execution_phase.phase values are additive. TypeScript consumers should keep default branches instead of assuming either union is permanently exhaustive.

Events with no interested subscriber or tracing exporter are discarded before payload enrichment and queuing; they are not buffered for future subscribers. Queued events capture their payloads when emitted, so later producer changes do not change delivery. Listeners share immutable plain-data snapshots; opaque values such as maps and typed arrays retain separate listener copies.

agent.commentary records completed preambles, not text deltas. It carries the original agent-event sequence and timestamp and attaches to the active harness span. Recent duplicate completions are suppressed per attempt. Like other queued diagnostics, commentary may be dropped under queue pressure; the session transcript remains the durable conversation record. At debug level, the diagnostic logger also records completion metadata without commentary text.

Model usage

model.usage is a trusted, in-process diagnostic event, not a JSONL log record. A representative event has this shape:

{  "type": "model.usage",  "ts": 1735689600000,  "seq": 42,  "provider": "openai",  "model": "gpt-5.4",  "channel": "webchat",  "agentId": "main",  "sessionId": "session-123",  "sessionKey": "agent:main:main",  "usage": {    "input": 120,    "output": 40,    "cacheRead": 30,    "cacheWrite": 10,    "promptTokens": 160,    "total": 200  },  "lastCallUsage": {    "input": 120,    "output": 40,    "cacheRead": 30,    "cacheWrite": 10,    "total": 200  },  "context": { "limit": 128000, "used": 160 },  "costUsd": 0.0012,  "durationMs": 850,  "trace": {    "traceId": "4bf92f3577b34da6a3ce929d0e0e4736",    "spanId": "00f067aa0ba902b7",    "traceFlags": "01"  }}

• ts is a Unix timestamp in milliseconds; seq is process-local.
• usage holds turn-level token counts. promptTokens includes input, cacheRead, and cacheWrite; lastCallUsage, when available, describes the final model call.
• context.used is the current prompt/context snapshot and can be lower than usage.total when cached input or tool-loop calls are involved.
• Provider/model/session identifiers, token buckets, lastCallUsage, context, costUsd, durationMs, and trace fields are optional. costUsd is an estimate and can be absent when model pricing is unavailable; it is not provider-reported billing. Trace context can also include parentSpanId.

The Gateway's /tmp/openclaw/openclaw-YYYY-MM-DD.log JSONL file and diagnostics.otel.logsExporter: "stdout" contain ordinary log records, not raw model.usage events. Public diagnostic subscriptions and diagnostics.stability do not expose trusted core usage events. The diagnostics-otel plugin converts them to metrics such as openclaw.tokens and openclaw.cost.usd and to openclaw.model.usage spans; those usage metrics and spans intentionally omit session identifiers.

Completed memory-flush and Skill Workshop review inference records its own usage and available cost estimate. These internal runs do not add their usage to the foreground conversation's totals.

For an external integration that needs session-correlated usage, query the authenticated Gateway instead:

openclaw gateway call sessions.usage --params '{"range":"30d","agentScope":"all"}' --jsonopenclaw gateway usage-cost --days 30 --all-agents --json

Both commands require operator.read. sessions.usage can include per-session sessionId, provider/model details, and token/cost summaries; per-session usage can be temporarily null while its cache refreshes. usage-cost provides aggregate estimates. Omit agentScope or --all-agents to scope the report to the default agent. For continuously updated clients, subscribe to session changes instead of polling usage reports. See the Gateway RPC method reference for usage methods and request options.

Message flow

• webhook.received / webhook.processed / webhook.error
• message.queued / message.processed
• message.delivery.started / message.delivery.completed / message.delivery.error

Gateway RPC

• gateway.rpc - trusted request observations with phases received, response, handler, and dispatch. Response outcomes are ok, error, unavailable, or suppressed; handler outcomes are returned or threw; dispatch outcomes are returned, threw, rejected, or cancelled. Dispatch records its response state (none, sent, unavailable, or suppressed) at settlement; a later response can still arrive. Durations and queue/admission semantics are described in Gateway RPC metrics.

Queue and session

• queue.lane.enqueue / queue.lane.dequeue
• session.state / session.long_running / session.stalled / session.stuck
• run.attempt / run.progress
• run.execution_phase (public, session-correlated embedded-runner startup milestones)
• diagnostic.heartbeat (aggregate counters: webhooks/queue/session)
• gateway.event_loop.sample (internal metrics-only completed window: intervalMs, delayMaxMs; no reader identity)

Harness lifecycle

• harness.run.started / harness.run.completed / harness.run.error - per-run lifecycle for the agent harness. Includes harnessId, optional pluginId, provider/model/channel, and run id. Completion adds durationMs, outcome, optional resultClassification, yieldDetected, and itemLifecycle counts. Errors add phase (prepare/start/send/resolve/cleanup), errorCategory, and optional cleanupFailed. With captureContent: true, started and completed events also carry bounded, redacted turn prompt and final assistant text as private listener-only data exported on spans, never on public payloads.

Exec

• exec.process.completed - terminal outcome, duration, target, mode, exit code, and failure kind. Command text and working directories are not included.
• exec.approval.followup_suppressed - stale approval follow-up dropped after a session rebound. Includes approvalId, reason (session_rebound), phase (direct_delivery or gateway_preflight), and the dispatcher timestamp. Session keys, routes, and command text are not included.