pi-gpt-voice

Voice and LAN remote control for Pi, without replacing your active model or tools.

Pi Voice uses a ChatGPT login for its realtime audio connection, then routes work into the current Pi session. The Pi session can use any model provider.

Requirements

• Pi 1.0.0 or newer, and Node.js 22.19 or newer.
• One of these platforms:
  • Linux on x86-64 with glibc 2.35 or newer, such as Ubuntu 22.04 or later, and the ALSA runtime library, libasound.so.2. Install libasound2 on Debian and Ubuntu (libasound2t64 on Ubuntu 24.04 and later), or alsa-lib on Fedora and Arch Linux.
  • macOS on Apple silicon (arm64). The helper is built for macOS 11 or newer; the maintainer has tested it on macOS 26. See macOS.
• A ChatGPT login for the realtime audio connection. Run /voice login, or keep an existing openai-codex login in Pi. See ChatGPT login.

Windows is not supported, and npm refuses to install the package there. npm cannot express which operating system and CPU pairs ship a helper, so it does install on Linux on Arm, Intel Macs, and musl-based distributions such as Alpine. Voice does not work there: Pi Voice reports that no helper is bundled for the platform, or, on musl, the Linux helper fails to start.

The package includes a prebuilt native audio helper, pi-codex-voice, for each supported platform. Automated checks build the Linux helper on Ubuntu 22.04 and the macOS helper on a GitHub macOS 15 arm64 runner, and confirm on each platform that it starts, exits cleanly, and creates a call offer without audio devices. They do not exercise microphones, speakers, or real calls, and the maintainer has not tested this release across a range of audio hardware.

Install

pi install npm:pi-gpt-voice

Restart Pi or run /reload, then run /voice. To try the package for one session without adding it to your settings, run pi -e npm:pi-gpt-voice. Review a package's source before you install it; Pi extensions run with your user permissions.

Do not load multiple copies of Pi Voice or other extensions that use the same commands, shortcuts, or LAN port. Disable conflicting extensions in pi config before you enable this one.

To update, run pi update npm:pi-gpt-voice and restart Pi or run /reload. If you previously installed Pi Voice from a local checkout, run pi remove <path-to-checkout> before pi install npm:pi-gpt-voice. Your settings in pi-gpt-voice.json stay in place. Settings from the original extension (pi-gippity-control.json) are not imported; configure Pi Voice again in /voice.

The source repository, giuseppecrj/pi-voice, is private, so the repository, issue, and homepage links in the npm metadata require access. The npm package contains the complete TypeScript and Rust source of each release.

Controls

Action	Default
Realtime voice	Ctrl+Alt+Space
Mute realtime microphone	Ctrl+Alt+M
Hold to dictate	Ctrl+Alt+D
LAN control server	Ctrl+Alt+G

Commands:

• /voice — settings
• /voice realtime
• /voice mute
• /voice dictation
• /voice stop
• /voice server — start the LAN control server; in the terminal it also shows a QR code to open it on a phone
• /voice create — plan and build a custom LAN web app
• /voice setup — configure audio devices
• /voice login — sign in to ChatGPT for voice

Tools:

• voice_stop — ends the active voice session the way /voice stop does, so you can say "hang up" or "end the call" during realtime voice. The voice assistant hands the request to Pi, and Pi calls the tool. The call ends while Pi is still finishing that turn, so Pi's final reply appears only in the transcript. With no active session, the tool reports that and changes nothing.

Settings live at <pi-agent-directory>/pi-gpt-voice.json, where the directory defaults to ~/.pi/agent and PI_CODING_AGENT_DIR overrides it. Keybind changes take effect after /reload.

Refresh realtime voice after compaction is off by default and requires a Voice context model. When enabled, it pauses at each successful compaction boundary, summarizes the compacted branch, and starts a fresh voice call without ending spoken mode. An initial summarization failure leaves the old call untouched.

Set lan.customWebApp: true to enable a custom main UI, then set lan.customWebAppPath. Use an absolute path for one global app in every Pi directory, or a relative path resolved from the active Pi session cwd for a project-specific app. It must point to a static directory containing index.html; the running server rereads it on refresh. lan.port is optional and defaults to 43120.

Companion extensions can register one built static app through registerVoiceRemoteApp. Pi Voice serves it under /_voice/apps/<id>/ alongside the main remote UI, replays its bounded app.state snapshot on browser reconnect, and forwards transient app.event messages through the existing mini-SDK. The app still uses VoiceRemote for activity, Pi events, prompts, drafts, voice, and reconnection; extensions must not start another server. While Pi waits on an extension prompt, activity enters waiting with the prompt title and the bundled UI shows Waiting for you.

The LAN server includes a microphone mute button. The host retains the Realtime WebRTC call and relays 24 kHz mono audio to the active browser, so moving between devices does not restart the voice session. The server uses a local HTTPS certificate, belongs only to the Pi session that started it, and stops when that session changes.

Each time the server starts, it creates a new secret access token. /voice server lists links that carry it as ?k=<token>, and the QR code holds the same link. Opening a link stores the token in a cookie and lands on the same address without it, so no page keeps the token in its URL. API requests (/api/...) and the client script (/_voice/client.js) are answered directly instead, so scripts keep ?k=<token> on each request. Requests without the token get a page that asks for the link, or a 401. Only the bundled app's manifest and icons are public, because browsers fetch them without cookies. A new server, for example after /voice server is toggled or Pi restarts, has a new token, so earlier links, bookmarks, and home-screen icons stop working; scan the new QR code. Treat a link like a password while its server runs: anyone who has it can control that Pi session. Browsers send a host's cookies to every port on it, so another HTTPS service on the same hostname would receive the token too; run Pi Voice on a machine with no untrusted HTTPS services. The certificate is self-signed, so accepting it does not prove which machine answered; use a network you trust. /voice create passes the token to the agent in the discovery URL, so it appears in that prompt and in the session.

The global realtime prompt lives at <pi-agent-directory>/REALTIME-SYSTEM-PROMPT.md; trusted projects can append .pi/REALTIME-SYSTEM-PROMPT.md. Pi Voice ships its current template and cumulative schema changelog as raw Markdown. It checks the marker only when realtime voice is engaged and tells you when to ask your agent to migrate an outdated customized prompt instead of rewriting it automatically. Both paths are shown in /voice.

Other Pi extensions can ask an active voice session to speak:

import { reportRealtimeVoicePrompt } from "pi-gpt-voice";

const announcement = {
	id: "my-extension:finished",
	prompt: "Briefly tell the user that the task finished.",
};
reportRealtimeVoicePrompt(pi, { ...announcement, active: true });
reportRealtimeVoicePrompt(pi, { ...announcement, active: false });

For an ongoing state, send active: true when it begins and active: false when it ends. For a one-off announcement, send both immediately as above.

macOS

The macOS helper uses Core Audio, or a PulseAudio server when one answers (see below). It is ad-hoc signed and not notarized. npm does not mark the files it installs as downloaded, so Gatekeeper does not prompt before the helper runs.

A Mac without a microphone, such as a headless Mac mini or Mac Studio, can use another machine's audio in two ways. Local realtime voice and dictation fail at start with failed to open the default microphone when the Mac has no input device and neither applies.

Audio over SSH

The macOS helper can play and record through a PulseAudio or PipeWire server on the machine you connect from, through a Unix socket that SSH forwards. When PULSE_SERVER=unix:<path> names a socket that answers, that server's default microphone and speaker become the defaults, and its devices appear in /voice setup next to the Mac's own. Otherwise the helper uses Core Audio.

From a Linux machine running PipeWire, forward its Pulse socket when you connect:

ssh -R /Users/<user>/.pi-voice-pulse.sock:$XDG_RUNTIME_DIR/pulse/native <mac-host>

Then start Pi on the Mac with PULSE_SERVER=unix:$HOME/.pi-voice-pulse.sock, or export it from the shell profile so that Pi started in tmux or Herdr panes inherits it. Shells and panes opened before you add the export keep their old environment: open a new one, or run exec zsh, and restart Pi. The forward lasts as long as the SSH connection; when it ends, the helper falls back to Core Audio at the next call. If the server stops answering for two seconds during a call, for example because the connection stalls, the call ends with an error instead of hanging.

Tools that keep their own SSH connection, such as Herdr's remote machines, read ~/.ssh/config rather than your command line. Give the forward its own host alias and use that alias only for the long-lived connection:

Host mac-voice
  HostName <mac-host>
  RemoteForward /Users/<user>/.pi-voice-pulse.sock /run/user/<uid>/pulse/native

With Herdr, add the machine as herdr machine add --label <label> mac-voice. Keep the forward off the alias you use for ordinary SSH. Each forwarding connection replaces the socket, and the Mac's SSH server leaves it in place, refusing connections, when that connection exits, so a short ssh would take the audio away from the long-lived one. With several forwarding connections open, the newest one owns the socket.

• The Mac's SSH server will not replace a socket that a previous connection left behind. Set StreamLocalBindUnlink yes in the Mac's /etc/ssh/sshd_config, or in a file under /etc/ssh/sshd_config.d/, or delete the socket before you reconnect.
• SSH creates the socket readable only by your user. Any process running as that user on the Mac can use the forwarded microphone and speaker while the connection lasts.
• PipeWire accepts the helper without a cookie. A PulseAudio server checks its cookie; copy it to the Mac and set PULSE_COOKIE. Only PipeWire has been tested.
• Over a network, echo cancellation sees more delay. Headphones avoid the model hearing itself.

The maintainer has tested device selection, microphone capture, speaker playback, a stalled connection, and a realtime voice conversation this way, from an Omarchy laptop with PipeWire 1.6 to a Mac Studio over Tailscale, with Pi running in a Herdr pane.

LAN control

Run /voice server on the Mac and open one of the links it lists, such as https://<mac-host>:43120/?k=<token>, on a phone or laptop. In the terminal, Pi lists every address and shows a QR code for the one a phone is most likely to reach: the tailnet name when Tailscale serving is on (below), then a private LAN address, then a Tailscale IP address, then the first listed. Scan it with the phone's camera. The browser supplies the microphone and speaker, and the realtime call stays on the Mac. The maintainer has tested LAN voice on a Mac Studio with no microphone, over Tailscale and the local network.

Serving over Tailscale

Turn on Control server over Tailscale in /voice settings, or set lan.tailscale: true, and the next /voice server also publishes the server with tailscale serve at this machine's tailnet name, such as https://studio.<tailnet>.ts.net/?k=<token>. Tailscale provides a certificate that browsers trust, so phones on the tailnet see no certificate warning, and the QR code uses this link. The access token is still required.

• It needs MagicDNS and HTTPS certificates turned on for the tailnet (Tailscale admin console, DNS page). On Linux, the user running Pi must be allowed to run tailscale serve, for example with sudo tailscale set --operator=$USER.
• It uses HTTPS port 443; set lan.tailscalePort to use another. Pi runs tailscale serve as a foreground process for as long as the server runs, and never edits the shared serve configuration. Tailscale refuses that process on a port any other entry uses, so it never replaces or removes anyone else's entry. If the port is taken, or Tailscale is unavailable, Pi shows a warning and keeps the LAN addresses.
• Tailscale removes the entry when its process ends: when the server stops, and when Pi exits or is killed, because the process ends when Pi's end of its stdin pipe closes. Nothing is left behind to clean up.
• Pi never asks for Funnel, and refuses a port where any configuration already allows it, because Tailscale applies that permission to every listener on the port, even one added later. So the address is reachable only from your tailnet. If tailscale serve stops on its own while the server runs, Pi drops the tailnet link and warns; the LAN links keep working.
• Other tailscale serve entries on the same tailnet name receive the access cookie, because browsers share a host's cookies across ports; Pi warns when there are any, so keep only services you trust there.

Permissions and firewall

macOS controls microphone access per app in System Settings > Privacy & Security > Microphone, and usually attributes the helper to the terminal app that runs Pi. Local microphone capture on macOS has not been tested. Grant the permission from a logged-in desktop session, because a permission prompt cannot be answered over SSH or on a locked display.

If the macOS firewall blocks incoming connections, allow them for Pi in System Settings > Network > Firewall.

ChatGPT login

Realtime voice and dictation need a ChatGPT login whose token the OpenAI realtime and transcription services accept. Pi's newer Sign in with ChatGPT on the openai provider does not qualify: OpenAI currently rejects its token for realtime calls and transcription.

/voice login fills in /login pi-voice; press Enter to open Pi's login dialog for the Pi Voice provider. Pi stores the credential in auth.json and refreshes it like any other login. The dialog offers:

• Browser login and Device code login, which use the same ChatGPT sign-in as the Codex CLI. This is the supported path.
• Experimental: Sign in with ChatGPT with extra scopes, which runs Pi's newer sign-in and asks for additional OAuth scopes to request. Use it only to test whether OpenAI has enabled voice for that login; the resulting token is sent to the OpenAI API instead of the ChatGPT Codex backend.

Voice uses the Pi Voice login when present and otherwise falls back to an openai-codex login. Both use port 1455 for the browser callback, so finish or cancel one login before starting another.

Voice diagnostics

Start Pi with --voice-debug to record how long each realtime call took to set up, and how a LAN browser played the call's audio. The flag is off by default:

pi --voice-debug

Each realtime call that becomes the active call, including an auto-resumed or post-compaction replacement, adds one pi-voice-setup-timings custom entry to the Pi session file. The entry holds only nonnegative millisecond numbers measured from the start of setup: offerCreatedMs, postStartMs, postEndMs, answerAppliedMs, dataChannelOpenMs, and helperSpawnToReadyMs (the helper's own spawn-to-ready time). Stages that were not reached are omitted. The entry contains no endpoint, headers, credentials, SDP, audio, transcripts, or error text. It is not sent to the model and does not invalidate the cached voice context summary. Cancelled, superseded, and failed calls add no entry, and a diagnostic failure never affects the call.

During a LAN call, the browser that holds the call reports its playback about every 2 seconds while audio plays, and each report adds a pi-voice-lan-playback custom entry. These entries are not sent to the model either. browser holds counts since the call started:

• underruns: times the playback queue ran dry. rebuffers counts the ones where audio resumed within 500 ms, which cut off speech; rebufferMaxMs is the longest such silence.
• gapsOver60Ms, gapsOver120Ms, gapsOver250Ms, and maxGapMs: gaps between audio chunks, under one second, as they arrived.
• droppedSamples and maxQueuedMs: audio discarded because more than 250 ms was queued, and the most that was queued.
• suppressedResets, chunks, receivedMs, sampleRate, baseLatencyMs, outputLatencyMs, and the browser's userAgent.

host holds the server's counts for the same socket: sentChunks, droppedChunks (skipped because a second of audio was already waiting to be sent), and maxBacklogBytes.

The browser also reports when its audio is paused, hidden, rerouted, or muted, as pi-voice-lan-audio-events entries. Each event has at (milliseconds since the call's audio started), a type, and usually a detail: context (the AudioContext state, such as running or interrupted), session and session-type (Safari's audio session, where available), visibility (visible or hidden), page (hide or show), devices (changed), mic (live, muted, unmuted, or ended), clock (stalled when the audio clock advances less than half as fast as wall time over a second spent wholly in the running state, then running again), timer (delayed when the page's once-a-second check ran at least three seconds late), and worklet (error if the playback worklet failed). The first events record the starting context, visibility, microphone, and session state. They come from the page itself, so they still arrive while playback is stalled; they wait until the call is active, and while the socket is backed up. Each detail must be one of its type's state names, or the host drops it.

Pi treats the word after an extension flag as the flag's value, so put --voice-debug after any message argument. Any value turns it on; omit the flag to turn it off.

Development

bun run check runs Biome, tsgo --noEmit, and bun test tests. The tests are synthetic: they use loopback servers, stubbed peers, and a scratch agent directory, with no audio devices, credentials, or remote network. Pi 1.0.0 runs extensions on Bun 1.3.14, so run the check on that Bun version as well as on your default one.

The published package is built by GitHub Actions, not from a local checkout. See RELEASING.md for the helper build, the third-party notices, and the release process.

Acknowledgements

Pi Voice is a fork of the pi-gippity-control extension by Igor Warzocha, published under the MIT License.

The native helper adapts audio processing code from OpenAI Codex under the Apache License 2.0. See src/voice/rust/VENDORED.md and src/voice/rust/vendor/openai-codex/.

License

MIT. LICENSE retains the copyright notice from the upstream package. THIRD-PARTY-NOTICES.txt and licenses/ cover the third-party code built into the native helpers, including the Rust crates, the bundled Opus codec, and the Rust standard library.