Wall time per run
Wall time is the sum of the active answering segments of a run: the session for HumanExam, each question's session for ZeroShot, each turn for MultiTurn.
Pooled models shows one column per arm, its models averaged with equal weight — the readable overview. Per model shows one column per model for every arm, grouped by arm, for how one arm moves across tiers. By model is the transpose: one group per model with every arm inside it, Corograph first, for the like-for-like read at each tier at a glance; the line under each group is Corograph against that ecosystem's bare harness. This choice applies to every figure on the site and is remembered.
Data behind this figure — ZeroShot + MultiTurn · both batteries
| arm | role | column | active answering time per run | runs | cells | min | max | sd | rank | runs in the repository |
|---|---|---|---|---|---|---|---|---|---|---|
| Corograph | Corograph | 9 models | 11.6 min | 330 | 34 | 4.4 min | 35.2 min | 4.5 min | 3rd of 9 | Haiku 4.5 · Sonnet 5 · Opus 4.6 · Opus 4.8 · Opus 5 · Fable 5 · GPT 5.6 Luna · GPT 5.6 Terra · GPT 5.6 Sol |
| CC Bare | bare harness | 6 models | 13.0 min | 201 | 22 | 1.9 min | 44.2 min | 7.9 min | 6th of 9 | Haiku 4.5 · Sonnet 5 · Opus 4.6 · Opus 4.8 · Opus 5 · Fable 5 |
| Codex Bare | bare harness | 3 models | 9.1 min | 120 | 12 | 3.8 min | 19.2 min | 3.9 min | 1st of 9 | GPT 5.6 Luna · GPT 5.6 Terra · GPT 5.6 Sol |
| Cursor Bare | bare harness | 8 models | 18.9 min | 320 | 32 | 4.4 min | 52.7 min | 9.4 min | 9th of 9 | Haiku 4.5 · Sonnet 5 · Opus 5 · GPT 5.6 Luna · GPT 5.6 Terra · GPT 5.6 Sol · Grok 4.6 · Composer 2.5 |
| Graphify | indexing tool on Claude Code | 6 models | 13.3 min | 201 | 22 | 1.9 min | 42.6 min | 7.7 min | 7th of 9 | Haiku 4.5 · Sonnet 5 · Opus 4.6 · Opus 4.8 · Opus 5 · Fable 5 |
| GitNexus | indexing tool on Claude Code | 6 models | 11.7 min | 201 | 22 | 5.4 min | 24.7 min | 4.0 min | 4th of 9 | Haiku 4.5 · Sonnet 5 · Opus 4.6 · Opus 4.8 · Opus 5 · Fable 5 |
| CodeGraph | indexing tool on Claude Code | 6 models | 9.1 min | 203 | 22 | 3.7 min | 17.3 min | 2.7 min | 2nd of 9 | Haiku 4.5 · Sonnet 5 · Opus 4.6 · Opus 4.8 · Opus 5 · Fable 5 |
| CodebaseMemory | indexing tool on Claude Code | 6 models | 12.3 min | 201 | 22 | 4.6 min | 25.3 min | 4.3 min | 5th of 9 | Haiku 4.5 · Sonnet 5 · Opus 4.6 · Opus 4.8 · Opus 5 · Fable 5 |
| Serena | indexing tool on Claude Code | 6 models | 13.7 min | 207 | 23 | 5.2 min | 37.5 min | 7.3 min | 8th of 9 | Haiku 4.5 · Sonnet 5 · Opus 4.6 · Opus 4.8 · Opus 5 · Fable 5 |
Figure 10. Wall time is the sum of the active answering segments of a run: the session for HumanExam, each question's session for ZeroShot, each turn for MultiTurn. Each column is the mean over valid, scored runs; the benchmark's own dispatch and capture time between segments is excluded. Timing is environment-coupled and reported as indicative magnitude only; see the disclosures below before comparing across the host boundary.
How this figure is computed
Definition. wall(r) = Σ duration of the run's answering segments, as recorded by the lane driver. Provider latency, lane parallelism and retry back-offs all ride this figure.
Aggregation. Cell means averaged with equal weight. Runs whose duration was not captured are excluded, never counted as zero.
Reading it. Lower is better, with the caveat that this is not a controlled latency benchmark.
Disclosures
The Corograph arm executed on the host through its gateway; the comparison group executed in containers on the same machine. Accuracy, tokens and tool counts are location-independent; wall time is not, and the figure is published for completeness rather than as a claim.
Runs executed in parallel lanes shared the host; per-lane contention is not modelled.
Provenance
Computed at build time from epochs/E004/summary/runs.json of the proof package (export tool tabench export public 0.1.29, scorer era tabench-1.1). Each row of the table above links to the run directories in the repository, where run.json, the answers, the verdicts and the transcript of every run can be read.