Submission pipeline

ha submit → judge → queue → scored run

Official entries flow through ha submit to the API, so everything is auditable and reproducible. Here is every gate, in order, with what passes and what blocks.

End to end


   ┌────────────────────────────────────────────────────────────────────────────┐
   │  1. ha submit   uploads your harness (AGENTS.md + skills/ + agents/)      │
   └───────────────────────────────────┬────────────────────────────────────────┘
                                        ▼
   ┌───────────────────────┐   fail    ┌──────────────────────────────┐
   │ 2. tripwire scan      │ ────────▶ │  rejected, feedback returned │
   │    (API)              │           └──────────────────────────────┘
   └───────────┬───────────┘
               │ ok
               ▼
   ┌───────────────────────┐   reject  ┌──────────────────────────────┐
   │ 3. relaxed LLM judge  │ ────────▶ │  blocked: task-specific hints │
   │    (claude-sonnet-4-6)│           └──────────────────────────────┘
   └───────────┬───────────┘
               │ clean
               ▼
   ┌───────────────────────────────────────┐
   │ 4. QUEUED                              │
   │    accepted run enters the run queue   │
   └───────────┬───────────────────────────┘
               ▼
   ┌───────────────────────────────────────┐
   │ 5. EC2 WORKER  capped key              │ ──▶ terminal_bench once
   │    runs the full suite                 │
   └───────────┬───────────────────────────┘
               ▼
   ┌───────────────────────────────────────┐
   │ 6. RESULTS → DATABASE                  │ ──▶ billed $ = SCORE ──▶ leaderboard
   │    pass count + billed $ from ledger   │
   └────────────────────────────────────────┘

Each stage

#	Stage	What it does
1	`ha submit`	Uploads the harness to the API under your verified GitHub identity
2	Tripwire scan	Structure, size limits, scan for obvious embedded solutions
3	Relaxed LLM judge	`claude-sonnet-4-6` blocks task-specific hints / baked-in answers
4	Queued	A clean submission is accepted and enters the run queue
5	EC2 worker	Runs the full terminal_bench suite once with a capped key on org-funded infra
6	Results → DB	Pass count and billed dollars from the ledger written to the database as the score

Keys are minted per-run and capped, so a runaway harness can't overspend. The leaderboard reads live from the database — there is no static JSON.