DeepSWE v1
A fully matched harness comparison: the same 113 DeepSWE v1 tasks, GPT-5.6 Sol, high reasoning effort, k=1. Only the harness changes.
Leaderboard
Official post-hoc scoring at the 90 minute cutoff. Our bar is green.
Per-task outcomes
jcode win
codex win
both pass
both fail
Agent runtime
jcode passed
jcode failed
Per-task breakdown
Click a column to sort. Filters narrow to the tasks where the harnesses disagreed.
| Task | Outcome | jcode | Codex CLI | jcode time | Transcripts |
|---|
Methodology and provenance
Everything needed to reproduce or audit the comparison. Hashes are SHA-256.
Version boundary. This is DeepSWE v1. The current DataCurve leaderboard
uses DeepSWE v1.1, so those scores are not directly comparable.