pi-llama-cpp-stats

Pi extension that shows real-time prompt processing statistics from llama.cpp's SSE stream.

Features

• Replaces the "Working..." text with a live progress bar during prompt processing
• Shows progress percentage, estimated time remaining, and tokens/second
• Shows the same bar while the context is being compacted (/compact, auto-compaction, overflow recovery) and while /tree summarizes a branch, then a live count of the tokens the summary is generating

Example display:

Prefilling... ████████░░░░░░░░░░░░ 40% · 30s · 1234 tok/s

During compaction Pi's own "Compacting context..." spinner owns the status row, so the prefill bar appears above the editor instead. When prefill finishes the line switches to counting the tokens being generated:

Auto-compacting... (esc to cancel)                                  ← Pi's spinner
Prefilling... ████████░░░░░░░░░░░░ 40% · 30s · 1234 tok/s          ← above the editor
Generating... 1.2k tok                                             ← after prefill

The live token count is estimated from the stream (llama.cpp only reports usage on the final chunk) and matches the exact figure within about 1% on prose. It clears when compaction ends, succeeds, fails, or is cancelled.

It disappears once prefill finishes (the summary is then generating, which the bar cannot measure) and when compaction ends, succeeds, fails, or is cancelled.

Installation

pi install npm:pi-llama-cpp-stats

Or install locally for the current project:

pi install npm:pi-llama-cpp-stats -l

Usage

1. Restart pi (or /reload)
2. Start a chat with a llama.cpp model
3. The "Working..." text will be replaced with a live progress bar during prefilling

Requirements

• llama.cpp with OpenAI-compatible API
• llama.cpp must be built with prompt_progress support (enabled by default in recent versions)