pi-tks

A live tokens-per-second (TPS) + time-to-first-token (TTFT) meter for pi's assistant streaming, rendered in the footer status line.

Tuned for local models (Ollama, LMStudio, …) — the case where throughput (tok/s), not just latency, is the number you actually care about. For cloud models you pay per token; for a local box the rate tells you how fast your hardware is going.

⚡ 45.3 tok/s · 1.2k tok                 ← live, while streaming
✓ 48.0 tok/s · prefill 812 tok/s · 1.2k tok · TTFT 0.3s  ← final (exact)
⏸ stalled · 1.2k tok                    ← silence longer than the stall threshold

Install

The extension is a single file plus a dependency-free core. No build step — pi loads .ts directly via jiti.

# install the published package from npm (recommended):
pi install npm:@nbafrank/pi-tks

# or try it for one invocation:
pi -e npm:@nbafrank/pi-tks

# or load a local checkout from this repo, run inside pi:
pi --extension ./src/extension.ts

Requires pi >= 0.84.0.

What it shows

• Live TPS — a 3-second sliding window over the streaming tokens (text, thinking, and tool-call arguments).
• Final tok/s — the decode rate: exact output tokens ÷ time from the first token to the end. Prefill time is excluded (that's what TTFT and prefill tok/s are for).
• TTFT — time from the provider request to the first token.
• Prefill tok/s — prompt tokens ÷ TTFT, the rate at which the model chewed through the input before the first token. Shown on the final line only, since the prompt-token count (usage.input) is only known at message end. Null (and hidden) when the prompt count is unavailable or TTFT is 0.
• Total tokens — live estimate, snapped to the provider's exact usage.output at message end.
• Stall detection — gaps longer than the threshold (tool execution, slow prefill) are excluded from the live rate, so the number reflects real generation speed, not idle time.

Command

/tps prints the last message's stats. /tps on / /tps off / /tps toggle enable/disable the footer meter.

How it works

The math is a small pure, pi-free engine in src/core.ts (TpsMeter + estimators) — the same shape as pi-webfetch's core. src/extension.ts is the thin pi wiring:

before_provider_request → fresh generation timer (T0 = TTFT origin)
message_update          → feed a cumulative token estimate (chars/4) per text/thinking/tool-call delta
message_end (assistant) → snap the estimate to the provider's exact usage.output

• Live estimate = accumulated generated chars ÷ 4 (robust across providers, no tokeniser needed).
• Final average is exact, because we reconcile to usage.output.
• Characters are counted from each stream event's delta, so tool-call arguments (often most of a coding agent's output) count toward the live rate, and *_start events don't start the TTFT clock early.

Develop

npm run smoke             # pure unit checks for the core (no pi, no network)
npm run typecheck         # typechecks core.ts + scripts under node types
npm run typecheck:extension  # links your pi install's types, typechecks extension.ts
node --experimental-strip-types scripts/harness-prefill.ts  # drives the real extension with a fake pi

License

MIT