agent-qa0.2.4
The self-improving Agentic QA harness with Memory
Untested on omp
This is a pi package. No one has reported whether it works on omp yet. If you try it, add your result below.
Reports 0
No reports yet.
Readme
English · 中文
Website · Docs · Demo · Blog · Changelog
Your AI QA engineer, with memory. Describe a user journey in plain language. agent-qa tests it on web, Android or iOS and keeps the evidence. Use the CLI, the dashboard, or your coding agent through MCP and skills.
- memidex: remembers facts and working procedures as reviewable files in your repository.
- Self-improvement: validates lessons from runs, retires stale knowledge, and uses your feedback to correct mistakes.
| Benchmark | Rank | Published result |
|---|---|---|
| AndroidWorld | #1 | 100% pass@1 (116/116 tasks) |
| AppControlBench | #1 | 99.6% completion (114.5/115 points) |
Read the full benchmark reports, with run evidence and methodology, on our blog. Both used gpt-6-sol with memory and self-improvement off; AppControlBench gives half credit for partial completion.
See it in action
Inspect a failure. The demo catches a cart showing $29 instead of $53, with the failed step and its screenshot side by side.
See what memory contributed. Inspect the knowledge supplied to a run and trace it back to the steps that used it.
Run from your terminal. Real output from the demo suite: checkout passes; the deliberate cart-total bug fails.
Get started
Requires Node.js 24+.
npm install -D agent-qa
npx agent-qa init
Follow the quickstart to connect a model and run the demo, or set up with your coding agent.
[!NOTE] Early stage. Expect bugs. Report an issue when something breaks.