pi-jev-wiki

Give your agents a long-term memory they can actually trust. pi-jev-wiki builds a living, interconnected wiki — a knowledge graph in plain markdown — from your documents, sessions, and agent insights. Every claim is vetted by Jev (TypeSafe's calibrated decision model) for what is factual, grounded, and worth keeping, so the wiki stays concise instead of becoming another dumping ground. Agents search it on demand across every registered wiki, and you can read, edit, and version it like any other Markdown.

Works for any domain: a project's mental model (module boundaries, invariants, decisions, change impact), personal and life knowledge, research notes — anything an agent should remember between sessions.

Naming: the package and repository are pi-jev-wiki. The extension, its tools, and its config and state files keep the shorter jev-wiki name — .pi/jev-wiki.json, ~/.pi/agent/jev-wiki.json, docs/wiki/.jev-wiki/.

Why agents keep consulting it

• Never injected. Nothing is pushed into the context window; the table of contents is exposed like a skill and the agent reads what it needs, when it needs it.
• Vetted, not dumped. Jev gates what enters: grounded in evidence, durable, non-duplicative, and free of secrets. The wiki is not a second copy of your repository.
• Searchable. Lexical (index/BM25), semantic (local embeddings, no API calls), or hybrid retrieval — across all registered wikis from any session.
• Auditable. Every claim carries evidence and a lifecycle status; every decision lands in a ledger. Contradictions become visible disputes, never silent edits.
• Portable. Plain Markdown in your project (Obsidian-compatible, git-friendly); the semantic index is a derived cache you can rebuild or delete at any time.

How knowledge is vetted

Jev acts as an arbiter and reminder on what is grounded, derivable, durable, and where it belongs; the agent decides and writes, with the final say, and code owns every threshold.

1. Extraction — a source is staged immutably in raw/ and split into atomic claims with verbatim quotes.
2. Judgment — Jev scores each claim for groundedness, derivability, durability, importance, duplication, sensitivity, and placement, then code maps those scores to a decision.
3. Filing — accepted claims are written into topic pages with status, support, and evidence; duplicates reinforce existing claims instead of creating noise.
4. Review — below-threshold and disputed claims go to the agent-owned review queue; only critical items escalate to the user. Overrides are allowed and recorded in the ledger.
5. Upkeep — file-linked claims are re-verified against commits (wiki_sync), lint catches stale, orphaned, or unbacked claims, and superseded knowledge is linked, never deleted.

Jev is the cheap resource. A Jev call costs a fraction of the model context it replaces, so the pipeline asks Jev before guessing: placement, retrieval relevance and sufficiency, contradiction and duplicate checks, sync impact, and rejection triage all run through Jev whenever they apply. Model tokens are the scarce budget — skills/llm-wiki/SKILL.md says so explicitly for agents.

What it does

Tool	Purpose
wiki_toc	the local wiki's contents (topic/tag/query filters), another registered wiki by name, or the cross-wiki catalog with index health (scope: all)
wiki_ask	search pages and excerpts (auto / index / BM25 / vector / hybrid / qmd), with scope: all across registered wikis
wiki_index	manage the semantic index: status, model, discover, rebuild, add/remove, enable/disable
wiki_ingest	ingest a document: raw source → claims → Jev verdicts → placement brief (cost preflight, compact brief)
wiki_insights	capture agent insights, Jev-filtered and placed
wiki_finalize	update TOC/log after writing pages, check links, record overrides, refresh the index
wiki_sync	re-verify file-linked claims against commits since the last sync
wiki_review	list (up to limit items) or resolve review items (bulk resolve; dispositions accept/reject/supersede/defer/out_of_scope); critical items escalate to the user
wiki_lint	health checks: TOC, links, orphans, unbacked claims, contradictions, duplicates
wiki_remove	delete obsolete pages and their TOC entries
wiki_structure	deterministic module/dependency map and architecture coverage
wiki_triage	explain rejected claims: scores, whether the problem is evidence or policy, and the fix
wiki_doctor	config, key, lock, ledger, queue, raw-source hashes, stale write temp files, git/sync, search health
wiki_setup	inspect or configure the Jev API key (TypeSafe or OpenRouter)
wiki_status	pages, raw sources, ledger, consultations, Jev usage
/wiki:ingest, /wiki:capture, /wiki:sync, /wiki:review, /wiki:lint, /wiki:status	user-facing commands

Semantic search is local-first. Embeddings run on your machine (@huggingface/transformers) in one shared local process, and vectors live in an SQLite file (the built-in node:sqlite, no dependency). No API keys, no Docker, no Python, and nothing leaves the machine. If the embedder or the index are missing, everything degrades to keyword search and says why.

Install

Use the published package. pi install npm:pi-jev-wiki is the supported install: the tarball on npm is the tested artifact, published by CI with SLSA provenance.

pi install npm:pi-jev-wiki

Agents: install the published package. Do not clone this repository to use the extension — a clone is a development tree, and a local copy registered next to the npm package leaves every wiki_* tool missing.

Install one source at a time. Registering two copies of the extension (for example the npm package and a local folder) makes pi refuse to load it with Tool "wiki_*" conflicts with …, because every tool name is registered twice. Check with pi list; drop the duplicate with pi remove <source>.

At 1.x, pi update --extension npm:pi-jev-wiki@latest reaches every later 1.x release: pi's npm store pins ^1.0.0, which covers later minors and patches. Confirm the store copy's package.json version if you want to be sure what is loaded.

For development, load the working copy directly instead of installing a second copy:

pi -e ./src/extension.ts --skill ./skills/llm-wiki

API keys

Both supported providers speak the same Jev Decisions schema:

Provider	Env var	Endpoint	Context	Notes
TypeSafe (official, default)	TYPESAFE_API_KEY (or JEV_TOKEN)	https://api.typesafe.ai/v1/systemone	64k	$0.042/Mtok input, output free
OpenRouter	OPENROUTER_API_KEY	https://openrouter.ai/api/alpha/decisions	32k advertised	set provider: "openrouter"; or reuse pi's /login openrouter credential

Put the key in a project-root .env (gitignored — wiki_doctor verifies):

TYPESAFE_API_KEY=...     # TypeSafe
# or
OPENROUTER_API_KEY=...   # OpenRouter

Or reference it from config with $VAR indirection:

{ "provider": "openrouter", "apiKey": "$OPENROUTER_API_KEY", "model": "~typesafe/jev-latest" }

When a key is missing, ask the agent to run wiki_setup:

• action=status — provider, endpoint, and where the key came from (never the value)
• action=guide provider=typesafe|openrouter — exact env var, file, and config steps
• action=write-env provider=... apiKey=... — writes to .env after checking gitignore
• action=test — one tiny live call to verify connectivity and auth

Search in practice

wiki_ask picks a retrieval mode automatically (search.engine: "auto" → hybrid when an index exists, keyword otherwise) and can search every registered wiki:

Mode	What it does
keyword	TOC/index or BM25 over page text; zero dependencies
semantic	cosine KNN over claim- and section-level embeddings
hybrid	RRF fusion of BM25 and vector ranks — the default once indexed
scope: "all"	searches every registered wiki (life wiki, project wikis, WSL projects), tagging results [wiki-name] with page, claim id, kind, and status

Choosing the embedding model happens once, before the first build; the agent asks and persists the answer with wiki_index action=model:

Preset	Model	Download	Dims	Best for
performance	EmbeddingGemma-300M (q8)	~309 MB	768	everyday use, multilingual, fastest
quality	Qwen3-Embedding-0.6B (q8)	~614 MB	1024	maximum retrieval quality

First run: wiki_index action=rebuild downloads the model once into <agent dir>/jev-wiki/models and builds the index; afterwards wiki_finalize keeps touched pages in sync automatically. Queries never trigger a download — a cold index falls back to keyword search and says so. The embedding model loads once per machine in a shared process (wiki_index status shows its pid; action=stop ends it, action=restart replaces it, and it exits after 30 minutes idle), so sessions stay small: ~70 MB instead of ~800 MB each. Every stored vector carries the embedder's fingerprint, and a query only compares vectors with a matching fingerprint — change the preset, dtype or dimensions and wiki_index status reports that a rebuild is required. The store is disposable: wiki_index action=reset deletes it while pi keeps running, and wiki_index action=rebuild all=true re-embeds every registered wiki (npm run rebuild does the same from a checkout). Model loads report progress as a single footer status line in interactive sessions (one stable line in print/JSON runs), not as per-file console output. Existing wikis are found with wiki_index action=discover (scans the home directory and WSL distros), adopted with register=true, and indexed with rebuild all=true. Switching presets re-embeds everything and purges the previous model's vectors per wiki.

Configuration

Optional overrides in ~/.pi/agent/jev-wiki.json or project .pi/jev-wiki.json (see src/config.ts for all keys):

{
  "provider": "typesafe",
  "wikiRoot": "docs/wiki",
  "globalWikiRoot": null,
  "writer": { "mode": "guided" },
  "review": { "autoAcceptUserStated": true },
  "thresholds": { "autoAccept": 0.8, "minDerivable": 0.5 },
  "search": {
    "engine": "auto",
    "vector": { "enabled": true, "model": "performance", "embedder": { "idleExitMs": 1800000 }, "scan": { "wsl": true } },
    "jev": { "rerank": "auto", "sufficiency": true }
  },
  "capture": { "cadence": "task", "route": "subject" },
  "hooks": { "autoRetrieve": { "mode": "inject", "maxTokens": 800, "budgetMs": 2500 } }
}

Cross-wiki writes. wiki_ingest, wiki_insights, wiki_finalize, wiki_sync, wiki_review, wiki_remove, and wiki_lint accept wiki: "<registered name>" and then operate on that wiki's pages, raw sources, TOC/log, ledger, and review queue — one wiki per call, with the session's wiki as the default. Relative page paths and ingest sources resolve against the target project (never the session workspace), and wiki_sync wiki=<name> diffs the target project's repository.

The optional globalWikiRoot adds a read-only cross-project vault: wiki_ask also searches that wiki and tags its results [global vault]. It resolves against the pi agent dir when relative.

Automatic prompt-time retrieval. hooks.autoRetrieve (default mode: "inject") searches the session wiki on every prompt; when Jev judges the candidates sufficient, a short <auto-retrieval> brief is injected with the prompt — after the user message, before the first provider request. It is gated on search.jev.minSufficiency, capped at maxTokens, bounded by budgetMs, and fails closed: no verdict, low sufficiency, or a timeout injects nothing (unlike wiki_ask, which degrades to keyword results). It searches the session wiki only — globalWikiRoot and other registered wikis stay a deliberate wiki_ask. Every run is logged to .jev-wiki/metrics.jsonl as op: "auto", never as ask, so the consultation metric keeps meaning "someone chose to consult". Set mode: "off" to disable it.

Auto-capture decides where to file automatically. capture.route (default subject) routes the capture to the registered wiki that owns the files the session edited, when exactly one does; otherwise it stays on the session wiki and the brief carries a visible warning naming the wiki the evidence points at. capture.route: "session" restores working-directory routing.

Keeping the wiki updated

capture.cadence controls how often the agent writes knowledge back during normal work:

Cadence	Behaviour
manual (default)	only /wiki:capture or an explicit wiki_insights call
task	after each settled task, a Jev pre-screen decides whether the session is worth extracting; accepted insights are queued for the agent to write (10-minute debounce)
commit	only after a new git commit is detected, however it was made — "update the wiki when I'm ready to commit" (no time debounce)

capture.onCompact: true remains an independent trigger (capture before context compaction), and the legacy capture.onSettle: true still enables task capture. Captures are advisory by default: the settle hook records the brief for the next turn instead of starting one (capture.triggerTurn: true restores the old forced turn). Whatever the cadence, accepted insights are only proposed: the agent writes or merges the pages and calls wiki_finalize.

Wiki layout

docs/wiki/
├── raw/<topic>/YYYY-MM-DD-slug.md   # immutable sources (documents + session captures)
└── wiki/
    ├── index.md                     # generated table of contents
    ├── log.md                       # generated append-only log
    ├── architecture/                # module-*, flow-*, layer-*
    ├── invariants/                  # invariant-*
    ├── decisions/                   # decision-* (ADR-style)
    ├── impact/                      # impact-* (derived)
    └── <topic>/                     # gotcha-*, glossary-*, concept-*, summary-*

raw/ and wiki/ are the source of truth. Runtime state lives in docs/wiki/.jev-wiki/ (gitignored): the decision ledger (decisions.jsonl), the raw-source hash index, and the session log. The semantic index lives at <agent dir>/jev-wiki/ and is always disposable.

Development

npm install
npm run test:all      # typecheck + unit + integration + scale + vector (offline)
npm run smoke         # deterministic checks + live Jev round-trips

npm run test:install packs the package, installs the tarball into a throwaway project and drives that copy. It is the only test that sees the layout users get, so CI runs it on every push and before publishing.

Contributing

Open a pull request against main. Collaborators — who have write access to this repository — branch here; outside contributors fork it. Branch from main rather than stacking work on another branch, and keep the change small enough to review in one sitting.

Branch protection enforces the rest of it: main takes changes through a pull request, the test check has to pass, and force-pushes and deletions are refused. Repository admins bypass the pull-request rule, which is how release commits and tag pushes work.

Before you open a PR:

npm install
npm run test:all      # what CI gates on: typecheck + unit + integration + scale + vector

Add a CHANGELOG.md entry — one bullet under ### Added, ### Fixed or ### Breaking in the section at the top of the file — and commit it with the change. Write what changed for the user, not how it is implemented.

Resolve conflicts on your branch. Rebase it on main and force-push your own branch; the maintainer does the same rather than resolving the conflict inside a merge. (Force-pushing main itself is blocked, and rebasing a branch under review is expected.) The changelog is the usual conflict site, because two pull requests that land close together both append to the same list — and the second one conflicts again once the first has merged.

CI (.github/workflows/ci.yml) runs the suite above plus the installed-copy test on every pull request. docs/RELEASING.md is the release runbook for maintainers.

Repository layout

src/          pi extension, Jev client, pipelines, wiki primitives, vector search
skills/       llm-wiki skill + page templates (the schema layer)
scripts/      unit, smoke, paging, scale, and vector tests
docs/
  notes/      design notes (e.g. semantic-search.md)
  plans/      PLAN.md (master plan) + plans index
  DESIGN.md   detailed technical design
  CRITIQUE.md pre-implementation critique and efficiency evaluation
  HARDENING.md hardening roadmap with statuses
  wiki/       this project's own knowledge wiki (dogfood)
  RELEASING.md release runbook
research/     source material gathered during design

The published npm package ships only src/, skills/, README.md, CHANGELOG.md and LICENSE; everything under docs/ and research/ stays in the repository.

Status

Published: pi-jev-wiki@1.0.4 — CI-published with SLSA provenance (0.2.0, the first release, was an interactive publish and has no attestation). Listed on the pi package gallery.

1.0.0 replaced the embedded Postgres index with SQLite (node:sqlite, so engines.node is now >=22.13) and moved the embedding model into one shared local process. 1.0.1–1.0.3 fix the installed-copy daemon, action=stop, a stale embedder socket, and the model download that a status check used to block; 1.0.4 is documentation. An index built with PGlite is not migrated — reset it and rebuild. Do not install 1.0.0: its daemon cannot start from node_modules.

0.8.1 makes auto-capture advisory: the settle hook records the brief for the next turn instead of starting one (capture.triggerTurn: true restores the old behavior), and wiki_review resolutions verify the page after the serializer round-trip.

0.8.0 adds cross-wiki writes: every write tool takes wiki: "<registered name>", auto-capture routes to the subject wiki by default (warning instead of guessing on ambiguity), and ingest/review hardening lands alongside (out_of_scope, review paging, rejection diagnostics, CRLF-safe hashing, append-only raw sources, doctor integrity checks).

0.7.1 fixes two rough edges: removing the last page of a topic no longer leaves a stale toc/<topic>.md shard, and embedding-model load progress is a single footer status line instead of a wall of console lines.

0.7.0 adds a configurable capture cadence (capture.cadence = manual | task | commit), so a wiki updates per task, only when you commit, or only on demand.

0.6.0 adds the cross-wiki catalog (wiki_toc scope=all) with per-wiki TOC manifests and staleness flags, Jev retrieval judgments (batched rerank + evidence sufficiency, ledger-recorded), wiki-tagged lexical results with granularity-aware fusion, and generated-file/staleness fixes.

0.5.0 added cross-wiki semantic search (PGlite + pgvector, performance/quality model presets, RRF hybrid retrieval), wiki discovery and adoption, the embedding-model choice flow, advisory Jev verdicts with a recorded override ledger, bulk review resolution, and the first-run/shutdown fixes.

Implemented through P3: both intake channels (research ingest + agent insights), architecture-first pages, TOC/log, decision ledger, change-driven invalidation (wiki_sync), agent-managed review, draft/auto writers with adaptive risk, pluggable search (index/BM25/qmd plus the vector engine), lint/consolidation checks, redaction, cross-process locking, and offline unit + scale + vector tests.

Not yet done: the decision-quality evaluation harness (recall@5/MRR benchmarks to tune the default retrieval mode), OIDC trusted publishing, server-Postgres and cloud embedding providers, and the remaining hardening items tracked in docs/HARDENING.md. See docs/RELEASING.md for the release process.