pi-prefill

A pi extension that shows a live prefill progress bar in the loading line. It works with any server that reports prefill progress.

A long prefill can take many seconds. Without feedback, the loading line looks idle. You cannot tell a slow prefill from a hang. pi-prefill fills that gap with a real progress bar.

What it shows

The bar tracks non-cached work only. Cached prompt tokens do not count as progress.

Part	Meaning
prefill	Label.
[██████░░░░░░]	Progress bar.
52%	Percent of uncached tokens processed.
3.4k tok/s	Instantaneous prefill rate.
gen in ~4s	Estimated time until the first token.
14.1k/27.3k	Processed / total prompt tokens.

You choose which parts to show. See Configuration.

Supported Engines

Engine	Status	Setup
llama.cpp	🟢 supported	none. llama.cpp is in providers by default
oMLX	🟢 supported from v0.7.1.dev1	add your provider id to providers
ExLlamaV3	🟢 supported via TabbyAPI	add your provider id to providers
Unsloth	🟢 supported	add your provider id to providers
Strata	🟢 supported	add your provider id to providers
Project Maya	🟢 supported	add your provider id to providers
vLLM	🟠 pending PR vllm#40371	add your provider id to providers

All engines need api: "openai-completions" on the model. That is the API that defines prompt_progress.

pi-prefill sends return_progress: true for listed providers only. All engines default that flag to false. An unlisted provider gets no progress data and shows no bar. pi-prefill never guesses.

Install

pi install npm:pi-prefill

Configuration

Add a prefill key to ~/.pi/agent/settings.json. It controls which parts the bar shows, in what order, and which providers are asked for progress.

{
  "prefill": {
    "views": ["label", "bar", "percent", "tokens", "tps", "eta"],
    "providers": ["llama.cpp", "exllama"],
    "significantTokens": 1024
  }
}

Valid view names: label, bar, percent, tokens, tps, eta. Default when unset: ["label", "bar", "percent", "tokens", "eta"].

providers lists provider ids that receive return_progress: true. Default: ["llama.cpp"]. Add your TabbyAPI or compatible-endpoint provider id to enable it there. Set it to [] to never modify a request. See Enabling progress.

significantTokens sets the token cutoff from Short prefills. Default: 1024. A whole number of 0 or more is valid. 0 disables the token gate, so the time gate alone decides.

pi reads this file on session_start. Restart pi after you change it.

How it decides

pi parses each stream chunk and forwards it to extensions through provider_stream_event. pi-prefill reads that event. It never inspects a host name, an IP address, or a baseUrl.

Server behaviour	Result
Stream carries prompt_progress	bar is shown
Stream has no prompt_progress	nothing is shown, pi's own loading line stays

The response decides. A cloud request, a proxy, or a server without progress support produces no output.

Short prefills

A bar that appears for 20 ms is noise. pi-prefill waits before it draws:

• uncached total above significantTokens (default 1024) tokens: draw at once;
• otherwise: draw only if prefill runs longer than 300 ms.

Cached tokens never count as progress.

Background cache warm

pi keeps provider prompt caches warm (cacheWarming setting, default "streaming"). A warm request is a real request, so it reaches the same hooks. With cacheWarming: "idle", a long warm prefill can draw a bar between your messages. Set cacheWarming to "off" if you do not want that.

Debug

Set PI_PREFILL_DEBUG=1 to log each inject, progress, and generation boundary:

PI_PREFILL_DEBUG=1 pi

The log goes to /tmp/pi-prefill-debug.log. Set PI_PREFILL_DEBUG_LOG to change the path.

License

MIT. See LICENSE.