Pi Model Doctor

English | 简体中文

Pi Model Doctor is a Pi extension that manages the model lifecycle in models.json. A normal pi install npm:pi-model-doctor installs it to the current user's global Pi settings; use -l only when you explicitly want a project-local installation. It discovers provider metadata from models.dev, fills in model capabilities, checks existing entries, and applies safe repairs without replacing user-owned settings.

Table of contents

• Install, update, and uninstall
• Global installation and project-local override
• Commands
• Safe writes and ownership
• Cache and offline behavior
• Provider capabilities
• Development

Install, update, and uninstall

Install from npm into the current user's global Pi installation (the default and recommended mode):

pi install npm:pi-model-doctor

This writes the package source to the user's Pi package directory and adds it to ~/.pi/agent/settings.json, so it is available in other projects for the same user. Install into the current project's .pi/settings.json only when you explicitly need a project-local pin:

pi install npm:pi-model-doctor -l

Update the installed package later with:

pi update npm:pi-model-doctor

Remove it from the same scope in which it was installed:

# Global/user installation
pi remove npm:pi-model-doctor
# Equivalent alias
pi uninstall npm:pi-model-doctor

# Project-local installation
pi remove npm:pi-model-doctor -l

After installing or removing a package, restart Pi or start a new session so the extension registration is rebuilt. Removing the package does not remove models already written to models.json; use /model-doctor remove <provider/model> for configuration entries, and keep the timestamped backup if you may need to roll back.

Global installation and project-local override

The repository's .pi/settings.json contains only this project's development tooling. It does not need a ../index.ts Model Doctor entry when the package is globally installed. Pi's global settings load the npm package through its package-root manifest:

{
  "packages": ["npm:pi-model-doctor"]
}

Use pi install npm:pi-model-doctor -l only for a project-local override; in that case Pi writes the package reference to .pi/settings.json for that project. Do not add both the global package and a local ../index.ts entry, or the extension may be loaded twice.

Development checkout

For this repository's development smoke test, the relevant entry in .pi/settings.json is:

{
  "extensions": ["../index.ts"]
}

Pi resolves relative paths in a project .pi/settings.json from the .pi/ directory. A ../index.ts entry would therefore point to this repository's root source, but it is not needed for normal global npm installation. For another Pi project, install the published package with pi install npm:pi-model-doctor, or add the package directory to that project's Pi settings. Installed npm/Git packages do not use ../index.ts; Pi reads the package-root manifest pi.extensions: ["./index.ts"] instead.

Why the implementation is now at the repository root

.pi/ is Pi's project-local runtime/configuration namespace, but a package intended for GitHub/npm/pi.dev should have its own package root. The implementation lives at the repository root so package.json, README.md, LICENSE, src/, tests, and the Pi manifest are packaged together. The project-local .pi/settings.json points to ../index.ts only for this repository's development smoke test; when installed from npm or Git, Pi resolves ./index.ts from the package's own pi.extensions manifest.

Commands

/model-doctor add <provider-id> [endpoint-url] [model] [--metadata-provider <models.dev-provider>] [--api <protocol>] [--api-key <reference>] [--allow-literal-api-key] [--dry-run] [--yes]
/model-doctor configure <provider-id> [endpoint-url] [model] [--api <protocol>] [--api-key <reference>] [--allow-literal-api-key] [--dry-run] [--yes]
/model-doctor list [provider]
/model-doctor check [provider/model]
/model-doctor fix [provider/model] [--dry-run] [--yes]
/model-doctor remove <provider/model> [--dry-run] [--yes]
/model-doctor refresh [--force] [--dry-run]
/model-doctor sync <provider-or-url> [--models <id1,id2>] [--metadata-provider <models.dev-provider>] [--api <protocol>] [--api-key <reference>] [--allow-literal-api-key] [--dry-run] [--yes]
/model-doctor migrate <provider/model> [--to <provider/model>] [--dry-run] [--yes] [--remove-source]
/model-doctor cleanup-backups [--keep <count>] [--max-age-ms <milliseconds>] [--dry-run] [--yes]
/model-doctor rollback <models.json.bak-timestamp> [--dry-run] [--yes]

sync reports metadata provenance per model in dry-run, write, and no-op notifications. It first reads the channel's advertised /models list for openai-completions and openai-responses (root URLs use /v1/models; explicit paths are preserved), then uses models.dev only for metadata. Channel IDs are exact and case-sensitive, including unknown IDs; explicit IDs absent from a successful channel list are rejected. Unsupported protocols and discovery failures report warnings and fall back to models.dev, then local/explicit IDs, labelled as unverified rather than live channel support. Publisher attribution is resolved per exact model ID even on these fallback discovery paths; an invalid explicit --metadata-provider fails rather than silently falling back. Requests resolve environment references and configured headers, never execute !command or resolve pi-auth: implicitly, reject redirects/private destinations except intentional loopback, and are bounded to 5 seconds, 1 MiB and 10,000 models. No documentation pages are fetched. Literal CLI keys are omitted from discovery unless --allow-literal-api-key is passed; stored channel keys remain authoritative. DNS preflight rejects private resolutions, but does not pin the subsequent HTTP connection: DNS rebinding remains a limitation, so use trusted configured channels. sync and lets the interactive UI select multiple models for this run via a paged, searchable picker: toggle individual models, select all or invert the search-filtered set, clear the selection, and page through large lists (10 per page) before choosing Done. Action controls (select-all/invert/clear, search, paging, Done) are listed before the model toggles so they stay visible in terminals without list scrolling. Canceling at any step exits with not-persisted and does not create a proposal or write. In non-interactive mode, pass --models model-a,model-b; there is no implicit catalog-first-model selection. Sync writes one combined proposal with one backup and one atomic write, so the selected models are applied together. sync --dry-run never writes models.json, backups, or caches. The interactive add model picker and migrate destination picker use the same paged, searchable pattern.

add always starts with a provider id: add <provider-id> [endpoint-url] [model]. The second positional value is an endpoint only when it is an HTTP(S) URL; otherwise it is the model id. URL-first syntax is not supported. If the endpoint is omitted, the provider must already exist in models.json, so its endpoint, API protocol, headers, authentication, and other transport fields remain authoritative. In the interactive UI, an omitted model selects from catalog candidates through the paged, searchable picker; non-interactive mode requires an explicit model id. A new channel can be created with add providerA https://gateway.example/v1 <model> or first initialized as provider-only with add providerA https://gateway.example/v1, then populated with add providerA <model> or sync providerA. An unlisted third-party channel can use an exact models.dev model as metadata-only. Metadata source precedence is explicit --metadata-provider, direct target metadata, exact publisher attribution from the reviewed official-source registry, then a unique catalog-only match. The registry selects a source only; model limits and facts remain dynamic catalog data. It currently verifies exact glm-5.3 attribution to Z.AI from Z.AI's GLM-5.3 docs and API reference; SDK package names alone never prove authorship. Dry-run, write, and no-op previews report source, reason, and persistence status. Official limits are references, not verified limits for your channel; provider pricing/cache behavior, endpoint, API, headers, and credentials are never copied as channel facts. If attribution is unavailable, the source is labelled catalog-only or conservative-default, and missing context/output values explicitly report Pi's conservative defaults. When the model id is ambiguous across models.dev providers, interactive mode offers a paged picker of the tied metadata providers (choosing one is equivalent to --metadata-provider), and the selection-required error lists the candidate provider ids for --metadata-provider <models.dev-provider>. configure providerA [endpoint-url] [model] changes the existing provider endpoint, API protocol, and/or API-key reference; without a model it also synchronizes the API on Model Doctor-managed models while preserving user-owned model APIs. Without --api, protocol inference uses existing configuration, catalog/provider metadata, endpoint evidence, and only then weak model-name hints; unresolved cases default to openai-completions. Supported API values are openai-completions, mistral-conversations, openai-responses, azure-openai-responses, openai-codex-responses, anthropic-messages, bedrock-converse-stream, google-generative-ai, google-vertex, and pi-messages. For a direct channel/model add, a bare root URL is normalized to /v1 only for openai-completions or openai-responses; existing paths, query strings, and fragments are preserved. Provider-only setup does not append /v1 because no model type is available. Provider website metadata is not fetched automatically. API credentials should be references such as $OPENAI_API_KEY, ${OPENAI_API_KEY}, !command, or pi-auth:provider; add --api-key itself authorizes storing a literal key and replaces any existing provider key; omitted credentials are preserved and references are stored unchanged without execution or expansion. --allow-literal-api-key remains compatible but is unnecessary for add; configure and sync still require it for literals. Secrets are never printed. Use --dry-run to inspect a proposal without writing. Non-interactive writes require --yes; --dry-run takes precedence over --yes. migrate accepts an explicit --to provider/model, or presents destination candidates in UI mode; non-interactive mode requires --to.

check works offline for the local file and uses the local models.dev cache when the network is unavailable; even without a catalog it reports local ownership, metadata, and header findings. The slash-command refresh forces a catalog read and reports full configuration findings without applying repairs; it never writes models.json, while the normal form may update catalog/policy caches. refresh --dry-run is fully read-only and also leaves catalog/policy caches unchanged. fix only changes fields owned by Pi Model Doctor. If a user explicitly changed an endpoint, header, compatibility object, or model capability, the change is reported as a conflict and is not overwritten. remove requires an exact provider/model target. migrate creates a destination model from current metadata, preserves safe user fields, reports endpoint/API/header conflicts without copying secrets, keeps the source by default, and requires --remove-source for explicit source removal; deprecated destinations are advisory and cannot be auto-applied. A no-op migration reports no changes and does not create a backup. Every mutation verifies that models.json did not change after proposal creation, then uses a backup, atomic write, persisted read-back verification, and automatic restoration if persistence verification fails. When the active Pi runtime uses the default agent models path, the command refreshes and verifies the model registry, reporting persisted-and-active, activation-failed, or persisted-reload-required; dry-run and cancellation report not-persisted. Backup cleanup is explicit only: /model-doctor cleanup-backups --keep <count> or --max-age-ms <milliseconds> previews/removes old timestamped backups after authorization; it never runs automatically.

Safe writes and ownership

The default file is join(getAgentDir(), "models.json") (normally ~/.pi/agent/models.json). Mutating commands:

1. Read the existing file.
2. Create a timestamped models.json.bak-<timestamp> backup when the file exists.
3. Merge only the requested provider/model.
4. Write through a temporary file and atomic rename.

Managed entries carry _piModelDoctor metadata with the source, last check, repair policy, managed fields, and the last managed values. Unknown JSON fields and user-owned headers/endpoint overrides remain intact.

Migration keeps the source model unless --remove-source is supplied. It does not copy API keys, OAuth, authorization/secret headers, endpoint overrides, or user-owned capability overrides; destination values win on conflicts. To roll back a persisted mutation, use /model-doctor rollback <models.json.bak-<timestamp>> [--dry-run] [--yes]. The command validates the backup as a regular, parseable models file, creates a separate safety backup of the current file, atomically restores the validated backup, and refreshes/verifies the active model registry when supported. If activation cannot be verified, reload Pi and run /model-doctor check; backups contain the original credentials and must be protected like models.json.

Cache and offline behavior

The local cache defaults to:

~/.pi/model-doctor/models-cache.json
~/.pi/model-doctor/providers-cache.json
~/.pi/model-doctor/policies-cache.json

policies-cache.json contains a versioned capability policy catalog used by the reasoning/cache resolvers. Its baseline records the Pi compatibility baseline (0.82.1), models.dev schema label (api.json), normalized schema version (1), observation date, PolicyCatalog schema version, and _piModelDoctor metadata version. Invalid, incompatible, sensitive, or insecurely-permissioned cache data is ignored and regenerated. models-cache.json stores the normalized complete catalog, while providers-cache.json stores provider summaries, environment-variable names, selected adapters, reasoning controls, and independent prompt/context/KV capability signals. Cache writes reject sensitive fields; the cache directory is mode 0700 and cache files are mode 0600. Capability results include resolved, partial, advisory, unsupported, or unknown status. Internally, missing prompt/context/KV evidence is null (unknown), not a denial; runtime models.json omits unknown cache diagnostics instead of serializing null/unknown. With no cache evidence, both cacheCapabilities and cacheResolution are absent; partial evidence emits only known subitems, without guessed negative controls/retention/affinity. Proposals and provider summaries retain tri-state evidence independently of this runtime projection; explicit false remains unsupported and explicit true remains a server capability even when the Pi adapter has no control field. Unknown capabilities never generate negative cache-control flags, and lack of an adapter control does not imply server non-support. Fallback runtime costs of zero are not pricing evidence and fallback confidence stays low; explicit catalog zero cache prices remain genuine pricing evidence. Pricing metadata is not treated as permission to enable runtime caching. sync can replace unchanged managed fallback metadata; fix can correct the prior generated fallback-denial fingerprint and intermediate null/unknown diagnostics only when its ownership snapshot still matches. Cleanup retains unrelated compatibility fields and extension controls, refreshes snapshots, and is repeatable without additional changes. Edited, unmanaged, or snapshot-free compatibility objects, including manual true/false controls, are preserved. Provider-summary cache schema is 2: old schema 1 summaries are ignored and regenerated; complete catalog and policy schema versions are unchanged. Explicit denial labels and nested supported: false take precedence over descriptive string hints; arbitrary non-empty labels are not support evidence. No external cache extension is required or configured automatically. Each result retains source and confidence so provider facts are not mistaken for verified Pi runtime behavior.

Override the location with PI_MODEL_DOCTOR_DIR; override the configuration target with PI_MODEL_DOCTOR_MODELS_PATH; override the models.dev endpoint with PI_MODEL_DOCTOR_MODELS_DEV_URL. A custom endpoint must pass the same HTTPS/private-host policy; explicitly trusted private test infrastructure requires PI_MODEL_DOCTOR_TRUSTED_ENDPOINT=1. Conditional refreshes preserve an existing ETag or Last-Modified validator when a successful response omits a replacement, and reject credential-like or newline-bearing response validators before caching. Refresh is local-first and failure-safe: a failed network refresh keeps a valid prior cache and reports that it is stale. A session-scoped background refresh runs every 24 hours by default with bounded random jitter to reduce multi-session refresh bursts; set PI_MODEL_DOCTOR_REFRESH_INTERVAL_MS=0 to disable it. PI_MODEL_DOCTOR_REFRESH_JITTER_MS can override the jitter bound. Background refresh only updates catalog caches and reports warnings; it never edits models.json.

Provider capabilities

The cache and reasoning engines are metadata-driven rather than a finite model table. They normalize prompt/context/KV-cache signals and reasoning toggle/effort/budget/adaptive options for OpenAI-compatible, OpenAI Responses, Anthropic Messages, Google, and unknown-provider fallback adapters. Provider-specific compatibility metadata records fields for reasoning budgets, thinking config, independent cache capabilities, and unsupported runtime behavior. Budget values are kept distinct from output-token limits, and unknown/advisory capabilities are never silently presented as enabled. New models.dev providers can therefore be discovered without changing the extension. For third-party channels, these are model facts and advisory capability data only; they do not prove that the third-party transport implements the same runtime behavior.

Development

From the repository root:

npm install
npm run typecheck
npm test
npm pack --dry-run

The repository deliberately does not install or pin @earendil-works/pi-coding-agent in devDependencies. It declares Pi as an optional host peer and the development scripts resolve the Pi installation already available on your machine. If Pi cannot be found automatically, set PI_HOST_PACKAGE to the installed @earendil-works/pi-coding-agent package directory. This keeps local checks aligned with the Pi version that will actually load the extension; it does not add the development helper to the published package.

The package can be installed into another Pi project with:

pi install npm:pi-model-doctor

Or tested from a local checkout with pi install /absolute/path/to/pi-model-doctor.

The domain modules are intentionally dependency-light and can be tested with temporary directories and mocked fetch implementations.