pi-cliffcompaction

Mechanical autocompaction for Pi and OMP. When the session hits Pi's compact trigger, this package does not call a model. It keeps the last few turns verbatim and replaces the rest with excerpts: truncate or drop, never rephrase, never compact a compaction.

TypeScript port of CliffCompaction (Nguyen, Cho, Chen, and Dettmers):

• Paper: CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents (PDF)
• Reference implementation: nguyenvuthientrang/cliffcompaction

Needs Pi 0.82+ (or OMP) and Node 22+. No Python runtime.

pi install npm:pi-cliffcompaction
omp install npm:pi-cliffcompaction

After install, fully restart Pi/OMP. /reload can keep old JavaScript modules loaded.

---

Compared to Pi compact

Default Pi compact asks a model to write a structured memo (Goal, Progress, Key Decisions, file lists). Tool dumps are truncated for that summarizer, then rewritten. The last ~20k tokens stay verbatim. The previous memo is fed into the next one.

This package keeps the same when (token window, overflow, /compact) and replaces what. Last 3 assistant-step turns stay raw. Older bulk is cut by content class. The previous cliff is discarded, not re-summarized.

	Regular Pi compact	CliffCompaction
Compact cost	Extra LLM call (latency + tokens)	Instant, no summarizer call
What the working model gets	A story of the session	Cuts of the session
Hallucination	Can invent progress or drop numbers	Cannot rephrase
Drift over many cliffs	Summary of a summary	Always from the live tail
Recent verbatim	~20k tokens	3 turns (configurable)
Long tool output	Summarizer may extract a fact from ~2k chars	Dropped if longer than 500 chars
/compact focus on X	Honored	Ignored (no LLM to instruct)
File lists (read / modified)	Carried in the memo	Not carried (hook summaries skip Pi's file tracker)
/tree branch summary	Default LLM	Untouched

Worth it for long agent loops whose context is mostly huge read / bash dumps, where the next turn needs the last few raw turns plus "what was called." That is the paper's workload.

Use regular compact instead when the fact you need later lives inside a long tool result (a signature, an error buried in 8k of log). The LLM memo can keep that sentence. Cliff will have dropped the dump. Also better if you want a human-readable "where were we?" or you pass instructions to /compact.

---

Install

From a clone of AdityaVG13/pi-stack, inside the repo:

pi install ./packages/pi-cliffcompaction
omp install ./packages/pi-cliffcompaction

Library only (no Pi extension):

npm install pi-cliffcompaction

import { compact, Engine, makeConfig } from "pi-cliffcompaction/lib";

Installing this package takes over session_before_compact. Pi still decides when to compact (compaction.reserveTokens / compaction.keepRecentTokens). This package decides what the summary contains and which suffix stays verbatim. Load only one compaction extension.

---

Use

Activation is automatic. There is no tool to call.

Trigger	What happens
Context crosses contextWindow - reserveTokens (Pi default reserve: 16384)	Mechanical cliff instead of an LLM rewrite
/compact	Same, on demand. Extra instructions are ignored
Provider overflow / length recovery	Same, with a keep-recent=1 ladder

Confirm it loaded:

/cliff status
/cliff config
/cliff reload

/cliff status prints the knobs and, after a cliff, the last event. The footer shows cliff · threshold · kept N (or overflow / manual).

A brand-new empty chat will not compact (nothing to gain; fail-open). Force a cliff with /compact after a few tool turns.

To restore Pi's LLM summarizer without uninstalling:

{ "enabled": false }

in the config file below, then /cliff reload (or restart).

---

How it works

Context grows append-only until Pi fires compact. Then:

1. Head (system + the first user/task messages before the first assistant turn) is preserved. Pi has no hole in the provider transcript, so that text is folded into the summary string.
2. Last keepRecent assistant-step turns stay verbatim (firstKeptEntryId).
3. The middle becomes one summary message, built by content class.
4. A previous CliffCompaction summary is dropped, not nested. The next pass compresses only original messages since the last kept boundary.

That is the cliff: a sharp drop to roughly the same floor after every compaction, with KV-cache reuse between cliffs.

Content classes

Defaults match the GitHub proxy, not the paper's 300-character thought cap. Set "thoughtMaxChars": 300 to match Algorithm 1.

Content	Treatment
Tool results	Kept iff length <= 500 chars (resultMaxChars), else dropped
Tool calls	One-line signatures: [name] {truncated args} (150 chars, cmdMaxChars)
Assistant text	Full by default (thoughtMaxChars: 0). Paper Algorithm 1 used 300
Thinking / reasoning	Kept as text by default; signatures / encrypted blocks are never re-sent. keepThinking: false drops it. Independent cap: thinkingMaxChars
Human text	Verbatim, sanity-capped at 20000 chars
Images	Dropped from summaries; still verbatim in head and recent turns
Prior summary	Dropped entirely

No auxiliary LLM call. If the mechanical pass cannot shrink the history, the handler returns and Pi's default path runs. Shadow mode cancels compaction so the original history is forwarded unchanged.

Escalation (overflow / strict)

If the default floor is still over budget:

1. keepRecent = 1
2. Cap assistant text at 300 and drop thinking
3. (strict) Truncate the summary itself, newest parts kept

---

Paper vs GitHub vs this package

Algorithm 1 in the paper is the research writeup. The GitHub proxy is the executable algorithm. This package gold-matches GitHub.

Piece	Paper Alg. 1	GitHub (gold)	This package
Keep-recent	last 2K messages (K turn pairs)	last K assistant-step turns (default 3)	GitHub
Thought cap	300 chars	0 = unlimited	GitHub; set thoughtMaxChars: 300 for the paper
Tool result	keep iff <= 500	same	same
Tool signature	150 chars	same	same
Prior summary	skip	skip	same
Head	messages[0], messages[1] in QUERY	everything before first assistant	same in compact(); Pi adapter folds head into the summary string
Never compact a compaction	discard previous cliff	drop previous summary; compact only live turns	same

Human text is not in the pseudocode loop. Section 2.2 of the paper says keep it verbatim. GitHub and this package do that.

Pi cannot keep a non-contiguous head+tail in the provider transcript. The Python proxy can leave original head messages as separate objects. The cut rules are the same; cache shape is not.

This is not a network proxy. It does not speak Anthropic/OpenAI HTTP, install launchd/systemd, or replace /tree branch summarization.

---

Config

Optional. Defaults match the open-source proxy (assistant text unlimited, keepRecent: 3).

Host	File
Pi	~/.pi/agent/cliffcompaction.json
OMP	~/.omp/agent/cliffcompaction.json

Override path: PI_CLIFF_CONFIG / OMP_CLIFF_CONFIG, or PI_CONFIG_DIR / OMP_CONFIG_DIR. CLIFF_* env vars override the file. See config.example.json.

{
  "enabled": true,
  "keepRecent": 3,
  "thoughtMaxChars": 0,
  "thinkingMaxChars": 0,
  "keepThinking": true,
  "cmdMaxChars": 150,
  "resultMaxChars": 500,
  "humanMaxChars": 20000,
  "shadow": false,
  "strict": false
}

Setting	Default	Meaning
enabled	true	false restores Pi's LLM summarizer
keepRecent	3	Verbatim assistant-step turns at the tail
thoughtMaxChars	0 (unlimited)	Cap on assistant text in the summary
thinkingMaxChars	0	Cap on thinking text (independent)
keepThinking	true	Fold thinking as text; false drops it
cmdMaxChars	150	Tool-call signature budget
resultMaxChars	500	Longer tool results are dropped
humanMaxChars	20000	Cap on summarized middle user text; not the protected live head
shadow	false	Cancel compaction; log what would have happened
strict	false	Walk summary truncation (rung 3) when still over the library threshold
thresholdTokens	200000	Engine/library trigger (chars/4). Pi's own trigger is separate

After edits: /cliff reload.

Tuning when cliffs fire is a Pi setting, not this file:

{
  "compaction": {
    "enabled": true,
    "reserveTokens": 16384,
    "keepRecentTokens": 20000
  }
}

This package then overrides the kept suffix to keepRecent turns, not keepRecentTokens.

---

Library

The Pi extension is a thin adapter. The algorithm is importable with no Pi host:

Export	Role
compact(messages, dialect, cfg)	Algorithm 1. Returns { messages, headLen, summary, cut } or null if there is nothing to gain. Kept messages are the original objects.
Engine.prepare(body, dialect)	Reference proxy pipeline: hash-chain, longest stored prefix, compact over thresholdTokens, store under the original chain hash.
Engine.reactive(ctx)	Context-length error ladder.
Dialects	anthropic, openai (Chat Completions), openai-responses, pi

Token estimates use the floor of serialized UTF-16 code units / 4, with Python-style separators but JavaScript number/string semantics. This is not a tokenizer. Images use PNG/JPEG/GIF/WebP dimensions (28x28 patches, capped at 4784); raw base64, data URLs and Pi image blocks are recognized. Unknown dimensions conservatively cost 4784 without fetching remote URLs.

Digests are implementation-local: key ordering, floating-point formatting and Unicode counts can differ from CPython. The shared ASCII reference fixtures do not establish general cross-language hash compatibility. Library inputs must be JSON data.

---

Invariants

• Summaries contain only excerpts of original text, never a paraphrase.
• Exactly one summary header after compaction; re-compaction does not nest.
• Head messages and kept tail messages are identity-equal to the input objects (library compact / Engine).
• Engine.prepare validates every message against its current serialized content, including nested in-place edits. Only identical content can reuse a digest/cost; hash-chain prefixes are reused only while canonical digests agree. The engine retains snapshots for its latest request, not every visited branch.
• Cached summaries are scoped to the dialect, configuration and fixed request budget. Emergency escalation/truncation stays request-local. Changed canonical content cannot reuse a summary covering the old content.
• Prefix-store entry and serialized UTF-8 byte caps are hard admission limits. Oversized entries are not cached and do not evict useful entries; zero capacity disables admission. These are not total heap limits.
• OpenAI Chat summary keys include the effective reasoning_content / reasoning text so reasoning redactions invalidate old summaries.
• Image token cost does not track base64 length.

Error model

• compact returns null when there is no assistant turn, not enough turns to keep, or the rewrite would not shrink the list.
• Engine.prepare never throws on a well-formed body; store misses and inconsistent entries fail-open to passthrough.
• The Pi hook catches handler errors and returns undefined (Pi default compaction). Protected heads containing images or other non-text content also use this fallback, rather than silently losing attachments. The fallback may use an LLM and does not preserve image bytes verbatim.
• Pi compaction keeps the original head across successive cliffs, without recycling an old summary or retaining the discarded middle. Protected head text is not trimmed, truncated or interpreted as a summary marker.
• Strict truncation packs against the actual local billable estimate, including escaping and the kept tail. Library head messages and the Pi hook's folded live head text are protected, even when that floor cannot fit.
• The engine exposes overBudget; the Pi hook persists details.overBudget with its estimate. Neither silently drops the protected task or tail to force a fit.

Tests

npm test --prefix packages/pi-cliffcompaction

Tests cover content-validated caching/redaction, warm/cold equivalence, escaped budget boundaries, cache admission, image accounting and protected Pi task text. tests/reference-gold.test.mjs checks the shared reference fixtures; tests/json.test.mjs records the JavaScript-local compatibility boundaries.

No-claim boundaries

• This package does not claim the paper's SWE-bench / Terminal-Bench / KernelBench scores. Those were measured on other scaffolds with this algorithm.
• Trigger timing is Pi's (compaction.reserveTokens). The GitHub proxy default of 200k tokens is a library default, not what Pi uses unless you set Pi's reserve so the remaining window matches.
• Selector / Soft Group Verification from the paper is out of scope.
• Model-level hash identity is conditional on collision freedom for the compared inputs. No global fixed-length hash injectivity, SHA-256 collision proof, or Lean-to-TypeScript refinement proof is claimed.
• Redacting a request prevents stale-summary reuse for that history; it does not securely erase older prefix-store entries from memory before eviction.
• Other extensions that also handle session_before_compact will race. Load one.
• /compact with extra instructions is ignored.
• Shadow mode on overflow cancels recovery compaction; the overflowing request is left as-is (fail-open).
• Long tool results older than keepRecent turns are gone. If the model starts re-reading files it already had, that is the expected miss, not a bug in the cut.

Citation

@article{nguyen2026cliffcompaction,
  title   = {CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents},
  author  = {Nguyen, Trang and Cho, Eulrang and Chen, Bingqing and Dettmers, Tim},
  journal = {arXiv preprint arXiv:2609.26779},
  year    = {2026}
}

License

MIT. Algorithm MIT from nguyenvuthientrang/cliffcompaction. Package source: AdityaVG13/pi-stack.