Codex Computer Use MCP

Expose OpenAI's official signed macOS Computer Use tools directly to Pi and MCP clients. The calling agent chooses each method and its arguments. This package runs no nested model and generates no action plan.

OpenAI does not produce or endorse this independent project. It relies on an experimental app-server API and installed ChatGPT components that may change.

Requirements

• macOS with an unlocked user session
• Node.js 22 or newer
• the official ChatGPT macOS app, by default at /Applications/ChatGPT.app
• the Computer Use component installed by ChatGPT, by default under ~/.codex/computer-use/
• Pi 0.99.0 or newer when using the Pi integration

macOS Screen Recording, Accessibility and TCC controls still apply.

Pi

Install from npm:

pi install npm:codex-computer-use-mcp

The extension registers the ten official Computer Use methods as Pi tools named computer_use_<method>, such as computer_use_get_app_state and computer_use_click. They use Pi's codemode exposure, so agents call and compose them from Pi's built-in codemode tool together with other Pi and MCP tools:

const state = await tools.computer_use_get_app_state({ app: "TextEdit" });
text(state.text);
image(state.screenshot);

await tools.computer_use_click({ app: "TextEdit", element_index: "7" });
await tools.computer_use_type_text({ app: "TextEdit", text: "hello" });
const next = await tools.computer_use_get_app_state({ app: "TextEdit" });
text(next.text);

The extension does not change which tools are active. Turn on codemode with "defaultTools": ["+codemode"] in settings.json or with pi --tools +codemode; Pi also turns it on when an MCP server uses codemode exposure. When neither codemode nor a Computer Use tool is active, the extension shows one notice at the first prompt.

computer_use_get_app_state resolves to { app, text, screenshot }, where screenshot is an image block for image() or null. It may return an accessibility-tree diff after the first inspection; pass disableDiff: true to request a fresh full tree. computer_use_list_apps resolves to the official text inventory produced by the app-server transport; unlike native @oai/sky, that transport does not provide the structured app array. Other methods resolve to their official text output. A failed call rejects with the official error text. Calls run one at a time, and calls after get_app_state reuse the signed session until the agent settles. The namespace instructions, available from describeNamespace("computer_use"), summarise this workflow.

Use /computer-use-status to inspect the installed component and transport status.

To run a source checkout:

npm ci
npm run build
pi -ne -e /absolute/path/to/codex-computer-use-mcp/integrations/pi/index.ts

MCP

Running the package binary starts a stdio MCP server with the same ten methods and computer_use_status:

npx codex-computer-use-mcp

Example Pi MCP configuration:

{
  "mcpServers": {
    "computer-use": {
      "command": "npx",
      "args": ["codex-computer-use-mcp"],
      "lifecycle": "lazy",
      "requestTimeoutMs": 180000,
      "directTools": false
    }
  }
}

Behaviour

The adapter has one mode. Pi and MCP both expose the ten official methods as typed tools; Pi gives them codemode exposure. Both are available without wrapper permission prompts. It adds no app allowlist, action gate, intent classifier, selector rewrite or focus policy. App-server uses Codex Full access. The adapter forwards any elicitation that the official host still emits.

Production calls require verified OpenAI-signed app-server and Computer Use binaries with Team ID 2DC432GLL2. The adapter uses an isolated, credential-free app-server context. It rejects any model-turn activity. Calls after get_app_state reuse the signed session, preserving element identifiers and official app state.

Closing a retained session sends a matching native turn-ended notification through the signed app-server before terminating its private processes. This lets the shared Computer Use service remove that session's software cursor without terminating the service. Cancellation and tool timeouts briefly wait for an outstanding native reply before notifying, so a late reply cannot recreate the cursor after cleanup. A stalled request, failed notification, or lost transport can prevent native cleanup. The call still returns its result, because the action already ran, and private process teardown still runs. brokerCleanupVerified describes private process cleanup, not visual confirmation that a cursor disappeared.

Audit records contain bounded metadata. They exclude arguments, app content, screenshots, prompts and credentials. MCP and CLI state defaults to ~/.direct-computer-use. Pi state defaults to direct-computer-use under the Pi agent directory. Set CODEX_COMPUTER_USE_HOME to override either default.

Paths

By default the adapter uses the Codex app-server at /Applications/ChatGPT.app/Contents/Resources/codex-cli/CodexCLI.app/Contents/MacOS/codex, falling back to /Applications/ChatGPT.app/Contents/Resources/codex on older ChatGPT builds. It uses the Computer Use app at ~/.codex/computer-use/Codex Computer Use.app, falling back to the copy bundled in older ChatGPT builds.

When ChatGPT is installed elsewhere or moves these files, set either path in config.json in the state directory:

{
  "codexPath": "/Applications/ChatGPT.app/Contents/Resources/codex-cli/CodexCLI.app/Contents/MacOS/codex",
  "computerUseAppPath": "/Users/me/.codex/computer-use/Codex Computer Use.app"
}

The adapter reads this file when it starts each Computer Use session, so changes apply without restarting Pi or the MCP server. Configured binaries must pass the same OpenAI signature and Team ID checks. When a component is missing, the error lists the paths checked and the config file to edit. /computer-use-status and codex-computer-use-mcp --status show the config path and the current error.

Pi limits returned text to 50KB or 2,000 lines. When text exceeds that limit, the complete text is written to a mode-0600 file in a private directory under /tmp. Images are returned directly and are never spilled to disk.

Development

npm ci
npm run check
npm run check:pi
npm test
npm run build

License

MIT