@runtypelabs/pi-otel0.1.7

@runtypelabs/pi-otel

Runtype telemetry for Pi agents. It turns every Pi run into a trace of GenAI semantic-convention spans with Runtype's runtype.* attributes, and sends it to Runtype over OTLP/HTTP. Each run then appears in your dashboard as an execution with its model, token usage, cost, tool calls, stop reason and transcript.

Pi has two runtimes with separate plugin systems, so the package has one shared core and one binding for each:

Entry point	For	How it attaches
@runtypelabs/pi-otel/coding-agent	The Pi coding agent (@earendil-works/pi-coding-agent)	A Pi package extension, configured from env variables
@runtypelabs/pi-otel/durable	A pi-durable harness (@earendil-works/pi-durable), including Cloudflare PiHarness	observePiDurable(harness, options)
@runtypelabs/pi-otel	The shared recorder, for a custom binding	createPiTelemetry(options)

• It has no runtime dependencies. It imports Pi's types only, builds OTLP/JSON itself and sends it with fetch, so it also runs on Cloudflare Workers without nodejs_compat.
• It only reads what Pi reports. It registers no Pi hooks that can change a request or a tool call, and it never throws into Pi.
• Every span names its producer: runtype.adapter.name is @runtypelabs/pi-otel, and pi.runtime is pi-coding-agent or pi-durable.

What Runtype shows

• Runs. One run per Pi run. In the coding agent, a run is one prompt until Pi settles (agent_settled), so retries and compaction stay in the same run. In pi-durable, a run is one input submission from placed to done or unanswered. An unanswered submission is a failed run.
• A trace tree. A root invoke_agent {agent} span, with one chat {model} child per model response and one execute_tool {tool} child per tool result.
• Tokens. Input, output, cache-read and cache-write tokens on each chat span, summed onto the run.
• Cost. Runtype prices the run on the server from gen_ai.response.model. When Runtype cannot price the model, it uses Pi's own catalog cost, which the package sends as runtype.total_cost.
• Transcript and eval capture. These need content (see Content). A run without content still shows its shape, tokens and cost, but Runtype cannot show the conversation or capture the run as an eval case.

Conversations group by gen_ai.conversation.id: the Pi session id in the coding agent, the conversation id in pi-durable.

Before you start

You need two values from Runtype:

• A Runtype API key with the TELEMETRY:WRITE scope.
• The id of the Runtype agent that owns the runs (agent_...). Create an external agent in the dashboard, or use the Connect page, which creates one for you.

Requires Node 22 or later, or a Cloudflare Worker.

The Pi coding agent

Install the package with Pi's package manager:

pi install npm:@runtypelabs/pi-otel

Then set these variables in the environment of the shell that starts Pi:

export RUNTYPE_API_KEY=rt_...            # a key with TELEMETRY:WRITE
export RUNTYPE_AGENT_ID=agent_...        # the Runtype agent that owns the runs
export RUNTYPE_CAPTURE_CONTENT=true      # optional: send the conversation
pi

Variable	Required	Meaning
RUNTYPE_API_KEY	Yes	A Runtype key with TELEMETRY:WRITE.
RUNTYPE_AGENT_ID	Yes	The Runtype agent id.
RUNTYPE_OTLP_URL	No	The full OTLP traces URL. Default: https://api.runtype.com/v1/otel/v1/traces.
RUNTYPE_CAPTURE_CONTENT	No	true sends prompts, answers, the system prompt and tool content. Any other value sends no content.
RUNTYPE_DEBUG	No	true writes setup and export errors to stderr.

• Content is off by default for the coding agent. A coding session carries source code and file contents. Without RUNTYPE_CAPTURE_CONTENT=true, Runtype gets the run's shape, tokens and cost, but no transcript and no eval capture.
• Without a key or an agent id, the extension does nothing. Set RUNTYPE_DEBUG=true to see why.
• To try it for one session without installing it, run pi -e npm:@runtypelabs/pi-otel.

The extension sends each model and tool span as it finishes, and the run's root span when Pi settles. When the session ends, it waits up to five seconds for delivery.

Configure the extension in code

If you start Pi through its SDK, you can build the extension yourself instead of using env variables. Content is on by default here, as in the durable binding:

import { createRuntypePiExtension } from '@runtypelabs/pi-otel/coding-agent'

const runtype = createRuntypePiExtension({
  agentId: 'agent_...',
  apiKey: process.env.RUNTYPE_API_KEY,
  content: { systemInstructions: false },
})

Pass runtype to Pi as an extension factory.

pi-durable in Node

npm install @runtypelabs/pi-otel

import { Harness } from '@earendil-works/pi-durable'
import { observePiDurable } from '@runtypelabs/pi-otel/durable'

const harness = await Harness.open(storage, { models, registry }, ctx)

// Attach before the first submission, so the observer sees every run from the start.
const observer = observePiDurable(harness, {
  agentId: process.env.RUNTYPE_AGENT_ID!,
  apiKey: process.env.RUNTYPE_API_KEY,
  harnessId: 'support-sessions.sqlite', // stable per storage; see below
  agentName: 'support',
})
// Re-adopt runs a previous process left unfinished, before any work resumes.
await observer.recover(ctx)

// ... submit work and wait for it ...

await observer.flush() // before the process exits
observer.dispose()

observePiDurable subscribes to the harness's committed records (subscribeCommits). It returns:

• recover(context): adopts the runs that were in flight when the previous process stopped, so they continue in the same Runtype run. Await it after Harness.open and before submitting or resuming work. A run it misses is still adopted on its next commit.
• flush(): waits for every export in flight. Call it before a process exits, or pass it to ctx.waitUntil in a Worker.
• dispose(): stops observing. A run that has not settled yet is not exported.

Cloudflare PiHarness

PiHarness (in agents 0.26.0) hosts pi-durable in a Durable Object. Attach the observer in the harness factory, right after Harness.open and before you return the harness:

import { createModels } from '@earendil-works/pi-ai/models'
import { createRegistry, Harness } from '@earendil-works/pi-durable'
import { observePiDurable, type PiDurableTelemetry } from '@runtypelabs/pi-otel/durable'
import { Agent } from 'agents'
import { PiHarness } from 'agents/harness/pi'
import { createAI } from 'agents/models/pi-ai'

export class Assistant extends Agent<Env> {
  ai = createAI({ binding: this.env.AI })
  registry = createRegistry()
  telemetry?: PiDurableTelemetry

  harness = new PiHarness({
    harness: async ({ storage, context }) => {
      const models = createModels()
      models.setProvider(this.ai.provider)
      const pi = await Harness.open(storage, { models, registry: this.registry }, context)
      this.telemetry = observePiDurable(pi, {
        agentId: this.env.RUNTYPE_AGENT_ID,
        apiKey: this.env.RUNTYPE_API_KEY,
        harnessId: this.ctx.id.toString(),
        agentName: 'assistant',
      })
      await this.telemetry.recover(context)
      return pi
    },
    defaults: { model: this.ai('@cf/zai-org/glm-4.7-flash') },
  })

  constructor(ctx: DurableObjectState, env: Env) {
    super(ctx, env)
    this.lifecycle.use(this.harness)
  }

  async ask(prompt: string) {
    const { text } = await this.harness.prompt(prompt)
    if (this.telemetry) this.ctx.waitUntil(this.telemetry.flush())
    return text
  }
}

• Store RUNTYPE_API_KEY as a Worker secret, not in wrangler.jsonc.
• Do not attach through this.harness.pi(). It resolves only after PiHarness has resumed Pi's work, so the observer would miss work that resumes after an eviction.
• A long run can continue across alarms and evictions. The observer sends each model and tool span as Pi commits it, with ids derived from the run. After an eviction, recover() rebuilds the run from Pi's stored entries without re-sending them, and the root span is sent when the submission settles. The root span carries the run's full token totals, iterations and final answer, rebuilt from storage. A child span whose export was still in flight at the eviction is missing from the trace tree.

Options

observePiDurable and createRuntypePiExtension take the same options. observePiDurable also requires harnessId: a stable identity for the harness's storage, such as ctx.id.toString() in a Durable Object or the SQLite file path in Node. pi-durable numbers conversations and submissions per storage, so without it two harnesses reporting to one agent would write into the same Runtype runs.

Option	Type	Meaning
agentId	string	Required. The Runtype agent that owns the runs.
apiKey	string	A Runtype key with TELEMETRY:WRITE. Without one, nothing is sent; batches still reach onBatch.
endpoint	string	The full OTLP traces URL. Default: https://api.runtype.com/v1/otel/v1/traces.
agentName	string	The name in invoke_agent {agentName}. Default: pi.
content	boolean | object	What conversation content to send. See Content.
serviceName	string	The service.name resource attribute.
fetch	typeof fetch	The fetch to send with.
onBatch	(body) => void	Receives each OTLP/JSON request body. Useful in tests.
onError	(error) => void	Receives export and observer errors. Errors never reach Pi.

createRuntypePiExtension also takes flushTimeoutMs (default 5000), the longest it waits for delivery when Pi settles or the session ends.

Content

Content is the conversation itself: the input messages, the final answer, the system prompt, and tool arguments and results. The durable binding and createRuntypePiExtension send all of it by default, because a run without it cannot become an eval case. The coding agent's env setup sends none of it unless RUNTYPE_CAPTURE_CONTENT=true.

// No content of any kind: shape, tokens and cost only. This also turns off eval capture.
observePiDurable(harness, { agentId, apiKey, harnessId, content: false })

// Turn off individual kinds, cap each value, and redact.
observePiDurable(harness, {
  agentId,
  apiKey,
  harnessId,
  content: {
    inputMessages: true, // gen_ai.input.messages
    outputMessages: true, // gen_ai.output.messages
    systemInstructions: false, // gen_ai.system_instructions
    toolArguments: true, // gen_ai.tool.call.arguments
    toolResults: false, // gen_ai.tool.call.result
    maxChars: 32_768,
    redact: (value, { kind }) => value.replaceAll(/sk-[A-Za-z0-9]+/g, '[redacted]'),
  },
})

• Kinds. Each of the five kinds is on unless you set it to false.
• maxChars. The longest value the package sends, measured after redact. The default is 65,536 characters and the limit is 262,144 (256 KiB), which is the largest attribute Runtype ingests. A value over the cap is dropped, not cut.
• redact(value, { kind }). Runs on every content value as a string before it is sent. Return the value, a replacement, or undefined to drop it. If it throws, the value is dropped.

Limitations

• The upstream APIs are not stable. pi-durable is experimental: its README says the API changes without notice between releases. PiHarness is a beta API in agents. The package reads only subscribeCommits, Pi's committed record shapes and pi-ai's message types, so changes stay small, but a new Pi minor can still break it.
• The pi-durable peer range is minor-locked. @earendil-works/pi-durable is an optional peer at ~1.0.0. The range widens only after the package's contract tests pass on the next minor. @earendil-works/pi-coding-agent is an optional peer at *, as Pi requires for packages the Pi host provides; the tests pin the version they run against.
• Use one instrumentation per run. Do not send the same Pi run to Runtype through two producers, for example this package and a future Pi or Cloudflare tracing integration. Runtype would get two invoke_agent spans for one run and count its tokens and cost twice.
• No subagent tree yet. Each pi-durable conversation reports its runs in its own trace. A child conversation is not linked to its parent.
• Timestamps come from Pi. Inside a Worker the clock does not advance during CPU work, so span durations can read close to zero.

License

MIT