cache-warm

Opt-in keep-alive for Pi's prompt cache. When enabled, the extension sends a tiny hidden turn before the cache TTL expires so a later user message is more likely to hit cache instead of paying for a miss.

Default is off. Installing the extension or starting a session does not send warm turns. Enable it for the current session with /cache-warm on, or open the interactive settings panel with bare /cache-warm in Pi's TUI. The panel clearly asks for consent before enabling billable warming; cancelling that prompt leaves warming off and writes no enablement preference. Keep-alive auto-stops after 30 minutes idle (from the last user turn or /cache-warm on). Set any duration with /cache-warm duration 1h (or 30m, 2h, forever).

This is a separate package from timestamp-pi. timestamp-pi only shows TUI timestamps and a countdown; it never sends keep-alive traffic. See docs/DECISIONS.md.

What it does

• Keep-alive pings — when the remaining prompt-cache TTL drops under 60 seconds (about 4 minutes after short-cache activity), and the session is idle with no queued messages, cache-warm injects a display: false custom message (Reply "." only. Do not use tools. #w <iso>-<id>) via sendMessage. The suffix is unique per send so providers do not see an identical prompt loop. A rate limit is on by default (15 pings per rolling hour, derived from the default four-minute cadence). Toggle with /cache-warm rate on|off or CACHE_WARM_RATE_LIMIT=off. Cap size is CACHE_WARM_MAX_PER_HOUR (0 or forever = unlimited while the limiter is on). Observed full one-hour Anthropic writes use a one-hour TTL; short and mixed writes schedule conservatively at five minutes. After 30 minutes with no user turn (configurable), keep-alive turns itself off so a forgotten session does not bill overnight.
• Tool isolation — once the hidden turn is confirmed, every hidden tool call is forcibly blocked, including retries, continuations, and interrupted work. If Pi drains queued user or foreign custom work before agent_settled, the guard is released at that message boundary so the external turn can use tools normally. The extension never changes the global active-tool list.
• Easy toggle — /cache-warm on and /cache-warm off work in every mode; bare /cache-warm opens the TUI panel (toggles outside the TUI)
• Honest metrics — attempts, successful refreshes, likely avoided misses, and estimated net USD saved

The five-minute fallback is an Anthropic heuristic, not a universal provider guarantee. The one-minute send margin is intentional: moving the ping to 4m50s would leave only ten seconds for event-loop, queueing, and provider latency and would make expiry more likely. Cache reads without new write evidence retain the most recent observed retention for the model/session. An unconfirmed dispatch is never retried within the same cache-activity epoch because a late first dispatch could otherwise create duplicate billed turns. /cache-warm on clears that suppression even when keep-alive is already enabled. Cache misses can still occur after the idle auto-stop, with an explicitly lower hourly cap, after cache-key-relevant context changes, or because of provider behavior.

Pings enter the LLM context. The assistant reply cannot be guaranteed invisible.

Commands

The TUI panel is select-driven: change the idle auto-stop and hourly rate settings, inspect status or metrics, or close it. Before enabling, confirmation shows the current idle limit and effective hourly cap and warns that billable prompts enter LLM context and replies may become visible. Increasing the idle limit or removing a cap while warming is active requires renewed confirmation. /cache-warm on and --cache-warm-enabled remain explicit opt-ins for scripts and startup use.

Startup flags and settings persistence

pi --cache-warm-enabled \
   --cache-warm-duration 1h \
   --cache-warm-rate off

Duration and rate edits are stored as private, validated per-setting JSON files in ~/.pi/agent/cache-warm-preferences/ (under Pi's getAgentDir()): activeMs.json and rateLimitEnabled.json. These files never store enabled state: a fresh runtime is always off unless the explicit --cache-warm-enabled flag opts into billable keep-alive for that session. Startup flags override stored duration/rate values for that session and are not written back. Malformed stored values fall back independently.

Metrics

Cost estimation uses @earendil-works/pi-ai pricing, including tiered rates and Anthropic one-hour cache writes. For OpenAI Responses requests, a validated reported/base cost ratio preserves flex or priority service-tier pricing in the counterfactual; unsafe or missing multiplier evidence reports N/A. The miss-billing heuristic is intentionally narrow: official OpenAI APIs are treated as ordinary uncached input; official Anthropic or explicit Anthropic-compatible cache control uses a five-minute cache write; a full observed cacheWrite1h uses a one-hour write. Mixed short/long writes are warmed on the shorter schedule but do not claim an avoided miss or estimated savings.

While enabled, the footer shows a short warm 4:32 · 2 hits · $0.041 line (countdown, avoided-miss hits, estimated net saved).

Install

Published on npm: cache-warm. Use Pi's package manager (pi install), not npm install alone.

pi install npm:cache-warm
pi install npm:cache-warm@0.2.0   # pin version
pi install -l npm:cache-warm      # project-local (.pi/settings.json)
pi -e npm:cache-warm              # one session, no install

Then run /reload in Pi (or restart it).

pi list
pi update npm:cache-warm
pi remove npm:cache-warm

From this repository (git):

pi -e ./extensions/cache-warm
pi install -l ./extensions/cache-warm

Keep-alive is off by default. Run /cache-warm on to enable it for the current session. It auto-stops after 30 minutes idle; use /cache-warm duration 2h (or CACHE_WARM_DURATION=2h) for a longer window. The hourly ping cap is on by default (/cache-warm rate off or CACHE_WARM_RATE_LIMIT=off to disable; CACHE_WARM_MAX_PER_HOUR defaults to 15).

Development

cd extensions/cache-warm
npm install
npm test

License

MIT — same as pi-extensions.