Terminal-Bench 2.1

89 terminal tasks, claude-opus-4-8, run through the Harbor harness on Modal. Every run is published with per-task cost, duration, and the complete trial transcript, failures included.

Versus the leaderboard

Published leaderboard entries use k=5 (5 attempts averaged). Ours is the best k=2 sweep cell, shown in green.

Accuracy against cost

Each point is one run: accuracy over all scored trials against cost per trial. The line follows reasoning effort at k=2. Dashed lines are competitor scores.

The medium cell is the best one: it beats xhigh on accuracy while costing less per trial and finishing tasks sooner. On this benchmark, more thinking stops paying for itself past medium.

Where the failures go

All runs

Click a column to sort. Each run links to its per-task breakdown and trial transcripts.