Make the agent cheaper without making it dumber.
Everyone runs the same harness — a vanilla piCLI on OpenRouter. You may add plugins, skills, agent profiles, and config. You may not touch pi's source. The benchmark is Terminal-Bench 2.x: 89 terminal-native tasks — fix a git repo, stand up a server, recover a database, and more. The baseline is vanilla pi + moonshotai/kimi-k3. Match or beat its pass count; among those who do, the lowest billed spend wins.
Nine allowlisted open-weight models. :free variants are banned, and all inference must go through the per-run key injected by CI.
From local tweak to a scored leaderboard run
Iterate locally
Install the `ha` CLI, run `ha login`, then `ha clone` the template repo. Tweak your harness — AGENTS.md, skills, agent profiles, config — and run `ha run --local` against a task subset with your own OpenRouter key + harbor. You see pass count and cost in seconds.
Submit
Run `ha submit`. Your identity is your verified GitHub login from `ha login` — one entry per person, no impersonation. The run is uploaded to the API; there is no PR.
API screens it
The API runs a tripwire scan and a relaxed LLM judge (Claude Sonnet 4.6) that blocks any task-specific hints or embedded solutions. Harness engineering only — no answers baked in. A clean submission is accepted and queued.
Queued + run
The accepted run is queued; an EC2 worker picks it up and runs the full terminal_bench suite once on org-funded infra with a capped key. Billed cost is pulled straight from OpenRouter's generation ledger — never self-reported.
Results + score
The worker writes results to the database: pass count and the billed dollars from the ledger. That dollar figure is your score, and it appears on the leaderboard read live from the DB. Cost is real billed spend — caching included — run on org-funded infra so nobody can fudge it.