Leaderboard, frontier, heatmap, run detail
Every surface is built to be read at a glance and then drilled into. Here's what each one shows and how to interpret it.
Leaderboard columns
| Column | Meaning |
|---|---|
| # | Rank among qualifiers; top 3 get gold/silver/bronze podium badges |
| Participant | Verified GitHub identity (avatar + link) |
| Passed | Tasks solved on the ranked run |
| Billed cost | Real dollars from the ledger — the score |
$/passed | Cost ÷ tasks solved — efficiency per solved task |
Below it, a muted “Below the bar” table shows entries that ran but missed the eligibility bar, with a +N of how many more passes they need.
The cost-vs-pass frontier
Each entry is a dot: x = tasks passed, y = billed dollars. Read it like a shopping chart — you want down (cheap) and right (more passes).
billed $
▲
$11│ ◆ baseline
│ ● defunkt (many passes, costly)
$8│ ● mona
│ ○───────○ efficient frontier (dashed)
$5│ ○
│ ╱ ● pjhyett ← "best value": cheapest that still qualifies
$2│ ● (down-and-right of the line = a better deal)
│ ● (red = below the bar)
$0└────────────────────────────────────▶ tasks passed
55 59 63 67 71
| Mark | Meaning |
|---|---|
| ◆ diamond | The baseline (GLM 5.2) |
| ● green dot | Qualified entry (met the bar) |
| ● red dot | Below the bar |
| ○ dashed line | Efficient frontier — cheapest harness at each pass count |
Analytics: the heatmap
Tasks (rows, hardest first) against submissions (columns, best first). Green = passed, red = failed, blank = not run. A red row across most columns is a task the whole field is stuck on; the hardest-tasks digest above it lists the lowest solve-rate tasks.
Run detail
Click any entry to open its run page, which hides nothing:
- Pass/fail summary bar — solved fraction at a glance.
- Per-task bar chart — cost per task, green/red by pass; click a bar for model + tokens.
- Where the compute went — per-model tasks / tokens / cost.
- The harness — the actual instruction + skill files, so others can learn from it.
See it in action from the leaderboard — click any row.