omp packages

@benjaminjamesxyz/pi-voice-input1.0.0

Voice-to-prompt extension for pi: mic capture -> neural denoising (clean_voice_dsp) + local Whisper transcription

benjaminjamesxyz Published 1 month ago SEE LICENSE ~173 downloads/mo

Untested on omp

This is a pi package. No one has reported whether it works on omp yet. If you try it, add your result below.

Did it work for you?

Result

One report per person. A new report replaces your old one.

Reports 0

No reports yet.

Readme

pi-voice-input

Voice-to-prompt extension for the pi coding agent. Speak into your microphone; a fully local pipeline denoises the audio, detects speech boundaries, transcribes it with Whisper, and drops the text straight into pi's prompt editor — or auto-sends it. No cloud, no API keys.

flowchart LR
    Mic[🎙️ Mic] --> Denoise[RNNoise denoise]
    Denoise --> VAD[Multi-cue VAD]
    VAD --> Whisper[whisper-rs]
    Whisper --> Pi[pi prompt editor]

Features

  • Single utterance — /voice: listen, transcribe, insert into the editor
  • Auto-send — /voice send: transcribe and submit as a prompt
  • Continuous dictation — /voice start / stop
  • Keyboard toggle — Alt+V
  • Agent tool — voice_input: lets the LLM ask you to speak
  • Barge-in — speaking during TTS playback silences the audio so the mic doesn't capture it
  • Live feedback — real-time status widgets (listening / speaking / transcribing)

Requirements

  • Linux with a working microphone (PulseAudio / PipeWire / ALSA, via cpal)
  • pi coding agent
  • Rust toolchain (the first build compiles whisper.cpp — allow a few minutes)
  • ~140 MB disk for the default Whisper model

Install

1. Download a Whisper model

mkdir -p ~/.cache/whisper
curl -L -o ~/.cache/whisper/ggml-base.en.bin \
  https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.en.bin
Model Size Trade-off
ggml-tiny.en.bin ~75 MB Fastest, least accurate
ggml-base.en.bin ~140 MB Good default
ggml-small.en.bin ~460 MB Best accuracy, slower

Any model path works via WHISPER_MODEL=/path/to/model.bin.

2. Build the native transcriber

The native/ directory vendors both Rust crates — no external path dependencies:

cargo install --path native/live_whisper_transcription

The binary lands in ~/.cargo/bin/live_whisper_transcription. For GPU acceleration, add --features cuda (also available: vulkan, metal, openmp).

3. Install the pi extension

pi install git:github.com/benjaminjamesxyz/pi-voice-input

Or symlink the extension file manually:

ln -s "$PWD/extensions/voice-input.ts" ~/.pi/agent/extensions/pi-voice-input.ts

If the binary isn't on PATH, point the extension at it with LIVE_WHISPER_BIN=/path/to/binary.

Usage

Command Action
/voice One utterance → editor
/voice send One utterance → auto-send
/voice start Continuous dictation
/voice stop Stop / cancel
/voice status Process + binary status
/voice clean Purge temp audio artifacts
/voice help Command overview
Alt+V Toggle capture

How it works

extensions/voice-input.ts           pi extension (TypeScript, loaded via jiti)
native/live_whisper_transcription   CLI transcriber: capture → DSP → VAD → Whisper
native/clean_voice_dsp              DSP library: RNNoise denoise, resampling, VAD

The extension spawns the binary with --ipc --no-save and consumes JSON events (ready, speech_start, segment, done, …) from stdout — a strictly typed bridge, compiled clean under tsc --strict.

Troubleshooting

  • "No Whisper GGML model file found" — no model in ~/.cache/whisper or ~/.local/share/whisper; set WHISPER_MODEL or download one (step 1).
  • "Binary: ❌ Not found" from /voice status — build step 2, or set LIVE_WHISPER_BIN.
  • Silence / no speech detected — check mic permissions and input device with arecord -l.

Development

npm install
npm run typecheck   # strict tsc --noEmit against pi's published types

License

To be added — native/clean_voice_dsp is dual-licensed MIT / Apache-2.0 (see LICENSE-MIT, LICENSE-APACHE).