Comparison · Epoch-4

Corograph vs Harnesses

The null hypotheses of each ecosystem — Claude Code with nothing added, OpenAI's Codex CLI, and Cursor Agent through its SDK — beside Corograph across every model any of them ran.

Every column is one model, so each read is harness against harness at a fixed model: the model is held constant and only the tooling around it changes.

armharnesscodebase indexHaiku 4.5Sonnet 5Opus 4.6Opus 4.8Opus 5Fable 5GPT 5.6 LunaGPT 5.6 TerraGPT 5.6 SolGrok 4.6Composer 2.5runs
CorographCorographCorograph codebase index596060606017606060——496
CC BareClaude Codenone364060306020—————246
Codex BareCodex CLInone——————606060——180
Cursor BareCursor Agent (SDK)none4040——40—4040404040320

Cells are valid scored runs at that model; 1242 in all. The indexing tools are not harnesses: each is installed on Claude Code, which stays the harness. Figures open with one column per arm (pooled models, each protocol × battery × model cell one vote); the Columns control switches every figure to one column per model for like-for-like reads at a fixed model, and the choice is remembered site-wide. The arm and model set is fixed to this comparison.

Figure 1 Context tokens per run

protocols
battery
model
columns

Pooled models shows one column per arm, its models averaged with equal weight — the readable overview. Per model shows one column per model for every arm, grouped by arm, for how one arm moves across tiers. By model is the transpose: one group per model with every arm inside it, Corograph first, for the like-for-like read at each tier at a glance; the line under each group is Corograph against that ecosystem's bare harness. This choice applies to every figure on the site and is remembered.

Context tokens per runlower is betterZeroShot + MultiTurn · both batteries
05.0M10.0M15.0MCC Bare 8.59MCodex Bare 4.55M★ 8.75MHaiku 4.5★ 3.37MSonnet 5★ 2.63MOpus 4.6★ 2.86MOpus 4.8★ 3.24MOpus 5★ 1.90MFable 5★ 4.35MLuna★ 2.85MTerra2.58MSolCorographCorograph8.59M6 modelsCC Bare4.55M3 modelsCodex Bare13.83MHaiku 4.59.22MSonnet 58.27MOpus 56.18MLuna3.30MTerra★ 2.47MSol★ 6.58MGrok 4.6★ 9.38MComp 2.5Cursor Barebare harness

Raw token counts are never pooled across vendors (tokenizers differ), so arms whose selected models span vendors are shown one column per model; ★ marks the best value at each model.

Data behind this figure — ZeroShot + MultiTurn · both batteries
armrolecolumncontext tokens per run
(input + tool results + cache read)		runscellsminmaxsdrank
(at this model)		runs in the repository
CorographCorographHaiku 4.5 ★8.75M3942.33M20.85M6.00M1st of 2Haiku 4.5
Sonnet 5 ★3.37M4041.54M7.77M1.87M1st of 2Sonnet 5
Opus 4.6 ★2.63M404786k7.38M1.96M1st of 1Opus 4.6
Opus 4.8 ★2.86M404884k7.66M2.13M1st of 1Opus 4.8
Opus 5 ★3.24M4041.34M5.90M1.52M1st of 2Opus 5
Fable 5 ★1.90M112793k3.17M1.01M1st of 1Fable 5
GPT 5.6 Luna ★4.35M404774k14.24M3.95M1st of 2GPT 5.6 Luna
GPT 5.6 Terra ★2.85M404603k8.14M2.44M1st of 2GPT 5.6 Terra
GPT 5.6 Sol2.58M404664k7.91M2.10M2nd of 2GPT 5.6 Sol
CC Barebare harness6 models8.59M201223.36M29.61M4.69M2nd of 2Haiku 4.5 · Sonnet 5 · Opus 4.6 · Opus 4.8 · Opus 5 · Fable 5
Codex Barebare harness3 models4.55M120122.15M10.61M2.03M1st of 2GPT 5.6 Luna · GPT 5.6 Terra · GPT 5.6 Sol
Cursor Barebare harnessHaiku 4.513.83M4042.32M23.84M5.11M2nd of 2Haiku 4.5
Sonnet 59.22M4045.15M16.79M3.26M2nd of 2Sonnet 5
Opus 58.27M4046.57M16.43M2.11M2nd of 2Opus 5
GPT 5.6 Luna6.18M4044.18M10.70M1.46M2nd of 2GPT 5.6 Luna
GPT 5.6 Terra3.30M4041.23M5.89M1.26M2nd of 2GPT 5.6 Terra
GPT 5.6 Sol ★2.47M404985k5.16M1.28M1st of 2GPT 5.6 Sol
Grok 4.6 ★6.58M4044.29M9.61M1.85M1st of 1Grok 4.6
Composer 2.5 ★9.38M4045.02M14.81M3.53M1st of 1Composer 2.5

Context tokens are the tokens the model read to answer a whole run: the uncached input plus every cache read, summed over every API call in the run, all threads included. Each column is the mean over the valid, scored runs of one arm at one model, every arm split by model because token counts are model facts; the dashed lines mark Corograph's and each bare harness's pooled value wherever pooling is licensed. Full figure, definition and disclosures →

Figure 13 Cost per correct answer

protocols
battery
model
columns

Pooled models shows one column per arm, its models averaged with equal weight — the readable overview. Per model shows one column per model for every arm, grouped by arm, for how one arm moves across tiers. By model is the transpose: one group per model with every arm inside it, Corograph first, for the like-for-like read at each tier at a glance; the line under each group is Corograph against that ecosystem's bare harness. This choice applies to every figure on the site and is remembered.

Cost per correct answerlower is betterZeroShot + MultiTurn · both batteries
$0.00$0.10$0.20$0.30Corograph $0.115CC Bare $0.308Codex Bare $0.093Cursor Bare $0.172$0.1159 modelsCorographCorograph2nd of 4$0.3086 modelsCC Barebare harness4th of 4$0.0933 modelsCodex Barebare harness1st of 4$0.1728 modelsCursor Barebare harness3rd of 4
Data behind this figure — ZeroShot + MultiTurn · both batteries
armrolecolumnUSD per correct answerrunscellsminmaxsdrankruns in the repository
CorographCorograph9 models$0.11533034$0.004$0.424$0.0862nd of 4Haiku 4.5 · Sonnet 5 · Opus 4.6 · Opus 4.8 · Opus 5 · Fable 5 · GPT 5.6 Luna · GPT 5.6 Terra · GPT 5.6 Sol
CC Barebare harness6 models$0.30820122$0.071$0.939$0.1854th of 4Haiku 4.5 · Sonnet 5 · Opus 4.6 · Opus 4.8 · Opus 5 · Fable 5
Codex Barebare harness3 models$0.09312012$0.004$0.240$0.0761st of 4GPT 5.6 Luna · GPT 5.6 Terra · GPT 5.6 Sol
Cursor Barebare harness8 models$0.17232032$0.005$0.652$0.1533rd of 4Haiku 4.5 · Sonnet 5 · Opus 5 · GPT 5.6 Luna · GPT 5.6 Terra · GPT 5.6 Sol · Grok 4.6 · Composer 2.5

Cost per correct answer divides what a run cost by the number of questions it answered correctly: the vendor-reported figure where the harness reported one, otherwise the benchmark's estimate from the published rate tables over the run's four token columns. Each column is the mean over valid, scored runs at one model (or pooled over the arm's roster in the pooled-models view); Corograph and the bare harnesses are the dashed references, and the data table names each row's cost basis. Full figure, definition and disclosures →

Figure 7 Outcome composition per run

protocols
battery
model
Outcome composition per runhigher is betterZeroShot + MultiTurn · both batteries
025507510093.3%Haiku 4.5Sonnet 5Opus 4.6Opus 4.8Opus 5Fable 5LunaTerraSolCorographCorograph · 330 runs1st of 489.4%−3.9%Haiku 4.5Sonnet 5Opus 4.6Opus 4.8Opus 5Fable 5CC Barebare harness · 201 runs3rd of 485.9%−7.4%LunaTerraSolCodex Barebare harness · 120 runs4th of 489.6%−3.7%Haiku 4.5Sonnet 5Opus 5LunaTerraSolGrok 4.6Comp 2.5Cursor Barebare harness · 320 runs2nd of 4
Data behind this figure — ZeroShot + MultiTurn · both batteries
armrolecolumnquestions correct · incorrect · unanswered, per runrunscellsminmaxsdrankruns in the repository
CorographCorograph9 models93.3%3303465.0%100.0%8.2%1st of 4Haiku 4.5 · Sonnet 5 · Opus 4.6 · Opus 4.8 · Opus 5 · Fable 5 · GPT 5.6 Luna · GPT 5.6 Terra · GPT 5.6 Sol
CC Barebare harness6 models89.4%2012240.0%100.0%11.5%3rd of 4Haiku 4.5 · Sonnet 5 · Opus 4.6 · Opus 4.8 · Opus 5 · Fable 5
Codex Barebare harness3 models85.9%1201255.0%100.0%13.6%4th of 4GPT 5.6 Luna · GPT 5.6 Terra · GPT 5.6 Sol
Cursor Barebare harness8 models89.6%3203255.0%100.0%11.4%2nd of 4Haiku 4.5 · Sonnet 5 · Opus 5 · GPT 5.6 Luna · GPT 5.6 Terra · GPT 5.6 Sol · Grok 4.6 · Composer 2.5

Every valid run becomes one thin column showing how its questions divided between correct (the arm's color), incorrect (dark) and unanswered or unjudged (black), so an arm's whole record is visible at once. Columns are ordered by model then repetition; the number above each group is the arm's mean accuracy and the number inside is its difference from Corograph. Full figure, definition and disclosures →

Figure 2 Accuracy per run

protocols
battery
model
columns

Pooled models shows one column per arm, its models averaged with equal weight — the readable overview. Per model shows one column per model for every arm, grouped by arm, for how one arm moves across tiers. By model is the transpose: one group per model with every arm inside it, Corograph first, for the like-for-like read at each tier at a glance; the line under each group is Corograph against that ecosystem's bare harness. This choice applies to every figure on the site and is remembered.

Accuracy per runhigher is betterZeroShot + MultiTurn · both batteries
406080100Corograph 93.3%CC Bare 89.4%Codex Bare 85.9%Cursor Bare 89.6%93.3%65.0%100.0%CorographCorograph1st of 489.4%40.0%100.0%CC Barebare harness3rd of 485.9%55.0%100.0%Codex Barebare harness4th of 489.6%55.0%100.0%Cursor Barebare harness2nd of 4
Data behind this figure — ZeroShot + MultiTurn · both batteries
armrolecolumnaccuracy
(% of questions correct)		runscellsminmaxsdrankruns in the repository
CorographCorograph9 models93.3%3303465.0%100.0%8.2%1st of 4Haiku 4.5 · Sonnet 5 · Opus 4.6 · Opus 4.8 · Opus 5 · Fable 5 · GPT 5.6 Luna · GPT 5.6 Terra · GPT 5.6 Sol
CC Barebare harness6 models89.4%2012240.0%100.0%11.5%3rd of 4Haiku 4.5 · Sonnet 5 · Opus 4.6 · Opus 4.8 · Opus 5 · Fable 5
Codex Barebare harness3 models85.9%1201255.0%100.0%13.6%4th of 4GPT 5.6 Luna · GPT 5.6 Terra · GPT 5.6 Sol
Cursor Barebare harness8 models89.6%3203255.0%100.0%11.4%2nd of 4Haiku 4.5 · Sonnet 5 · Opus 5 · GPT 5.6 Luna · GPT 5.6 Terra · GPT 5.6 Sol · Grok 4.6 · Composer 2.5

Accuracy is the share of a run's questions whose answer the grading pipeline marked correct, so every dot is one complete run of one arm at one model. The solid tick is the arm's pooled mean, the thin line its worst-to-best range, and dots are translucent so overlap reads as density; hover a dot for its model, repetition and run directory. Full figure, definition and disclosures →

Figure 14 Cost per run

protocols
battery
model
columns

Pooled models shows one column per arm, its models averaged with equal weight — the readable overview. Per model shows one column per model for every arm, grouped by arm, for how one arm moves across tiers. By model is the transpose: one group per model with every arm inside it, Corograph first, for the like-for-like read at each tier at a glance; the line under each group is Corograph against that ecosystem's bare harness. This choice applies to every figure on the site and is remembered.

Cost per runlower is betterZeroShot + MultiTurn · both batteries
$0.00$5.00$10$15Corograph $2.51CC Bare $6.15Codex Bare $1.80Cursor Bare $3.66$2.51$0.097$7.27CorographCorograph2nd of 4$6.15$2.14$15.02CC Barebare harness4th of 4$1.80$0.110$4.08Codex Barebare harness1st of 4$3.66$0.141$12.86Cursor Barebare harness3rd of 4
Data behind this figure — ZeroShot + MultiTurn · both batteries
armrolecolumnUSD per runrunscellsminmaxsdrankruns in the repository
CorographCorograph9 models$2.5133034$0.097$7.27$1.612nd of 4Haiku 4.5 · Sonnet 5 · Opus 4.6 · Opus 4.8 · Opus 5 · Fable 5 · GPT 5.6 Luna · GPT 5.6 Terra · GPT 5.6 Sol
CC Barebare harness6 models$6.1520122$2.14$15.02$2.654th of 4Haiku 4.5 · Sonnet 5 · Opus 4.6 · Opus 4.8 · Opus 5 · Fable 5
Codex Barebare harness3 models$1.8012012$0.110$4.08$1.281st of 4GPT 5.6 Luna · GPT 5.6 Terra · GPT 5.6 Sol
Cursor Barebare harness8 models$3.6632032$0.141$12.86$3.053rd of 4Haiku 4.5 · Sonnet 5 · Opus 5 · GPT 5.6 Luna · GPT 5.6 Terra · GPT 5.6 Sol · Grok 4.6 · Composer 2.5

Every dot is what one run cost (vendor-reported where available, otherwise estimated from the rate tables), so the spread of an arm's price is visible, not just its mean. The solid tick is the pooled mean and the thin line the range; hover a dot for its model, repetition and run directory. Full figure, definition and disclosures →

Figure 15 Tool result tokens per run

protocols
battery
model
columns

Pooled models shows one column per arm, its models averaged with equal weight — the readable overview. Per model shows one column per model for every arm, grouped by arm, for how one arm moves across tiers. By model is the transpose: one group per model with every arm inside it, Corograph first, for the like-for-like read at each tier at a glance; the line under each group is Corograph against that ecosystem's bare harness. This choice applies to every figure on the site and is remembered.

Tool result tokens per runlower is betterZeroShot + MultiTurn · both batteries
0500k1.0M1.5MCC Bare 125kCodex Bare 240k170k59kHaiku 4.580k21kSonnet 576k26kOpus 4.666k23kOpus 4.897k42kOpus 548k21kFable 5269k119kLuna186k77kTerra178k97kSolCorographCorograph125k3kCC Bare240k35kCodex Bare142k42kHaiku 4.574k19kSonnet 596k25kOpus 5301k44kLuna453k50kTerra718k52kSol210k66kGrok 4.6248k89kComp 2.5Cursor Barebare harness

Raw token counts are never pooled across vendors (tokenizers differ), so arms whose selected models span vendors are shown one column per model; ★ marks the best value at each model.

Data behind this figure — ZeroShot + MultiTurn · both batteries
armrolecolumntool result tokens per run
(estimated)		runscellsminmaxsdrank
(at this model)		runs in the repository
CorographCorographHaiku 4.5170k39459k292k83k2nd of 2Haiku 4.5
Sonnet 580k40421k151k43k2nd of 2Sonnet 5
Opus 4.6 ★76k40426k132k34k1st of 1Opus 4.6
Opus 4.8 ★66k40423k101k25k1st of 1Opus 4.8
Opus 597k40442k162k40k2nd of 2Opus 5
Fable 5 ★48k11221k77k22k1st of 1Fable 5
GPT 5.6 Luna ★269k404119k413k82k1st of 2GPT 5.6 Luna
GPT 5.6 Terra ★186k40477k269k57k1st of 2GPT 5.6 Terra
GPT 5.6 Sol ★178k40497k276k48k1st of 2GPT 5.6 Sol
CC Barebare harness6 models125k201223k536k147k1st of 2Haiku 4.5 · Sonnet 5 · Opus 4.6 · Opus 4.8 · Opus 5 · Fable 5
Codex Barebare harness3 models240k1201235k496k156k2nd of 2GPT 5.6 Luna · GPT 5.6 Terra · GPT 5.6 Sol
Cursor Barebare harnessHaiku 4.5 ★142k40442k617k112k1st of 2Haiku 4.5
Sonnet 5 ★74k40419k156k45k1st of 2Sonnet 5
Opus 5 ★96k40425k370k74k1st of 2Opus 5
GPT 5.6 Luna301k40444k883k222k2nd of 2GPT 5.6 Luna
GPT 5.6 Terra453k40450k1.29M280k2nd of 2GPT 5.6 Terra
GPT 5.6 Sol718k40452k1.22M368k2nd of 2GPT 5.6 Sol
Grok 4.6 ★210k40466k585k132k1st of 1Grok 4.6
Composer 2.5 ★248k40489k600k146k1st of 1Composer 2.5

Tool result tokens are the volume of text the arm's tools pushed back into the model's context over a run — file contents, search hits, shell output, or an index query's result — estimated per call from the result size. Every dot is one run, split per model for every arm; the solid tick is the mean and the thin line the range. Full figure, definition and disclosures →

Figure 12 Pass rate by question category and difficulty

protocol
battery
model

Each cell pools the arm's Anthropic models with equal weight per model, so every arm is compared over the same vendor's models; an arm that ran none of them shows a gap.

Pass rate by question category and difficultyhigher is betterZeroShot · both batteries · Anthropic models
absence-proofaggregationcallscollisioncross-stackdbdispatchendpointsfield-grainimpactmulti-hopnameless-entryreality-checktypesdifficulty 1difficulty 2difficulty 3difficulty 4CorographCorograph828496929999991006798911009798100989891CC Barebare harness766089569696849463828889849498948877Cursor Barebare harness8370986299100951008097981009398100989887−36 pts+36 ptscolor = difference from bare Claude Code in the same column · number = pass rate %
Data behind this figure — ZeroShot · both batteries · Anthropic models
armcolumnpass rateattemptscorrectmodels
cg-ask-fz-009absence-proof82.0%100825
cg-ask-fz-009aggregation84.0%100845
cg-ask-fz-009calls96.0%4484296
cg-ask-fz-009collision92.0%100925
cg-ask-fz-009cross-stack98.5%4484416
cg-ask-fz-009db99.2%4484446
cg-ask-fz-009dispatch99.0%100995
cg-ask-fz-009endpoints99.8%5045036
cg-ask-fz-009field-grain67.0%100675
cg-ask-fz-009impact98.1%3363296
cg-ask-fz-009multi-hop91.0%100915
cg-ask-fz-009nameless-entry100.0%1501505
cg-ask-fz-009reality-check97.2%2502435
cg-ask-fz-009types98.0%2802746
cg-ask-fz-009difficulty 1100.0%5605606
cg-ask-fz-009difficulty 298.0%5605486
cg-ask-fz-009difficulty 397.5%5605456
cg-ask-fz-009difficulty 490.8%10009085
coldabsence-proof76.0%100765
coldaggregation60.0%100605
coldcalls89.3%3603185
coldcollision56.0%100565
coldcross-stack96.3%3603455
colddb96.3%3603455
colddispatch84.0%100845
coldendpoints93.8%4053775
coldfield-grain63.0%100635
coldimpact81.7%2702165
coldmulti-hop88.0%100885
coldnameless-entry88.7%1501335
coldreality-check84.4%2502115
coldtypes94.4%2252115
colddifficulty 198.4%4504425
colddifficulty 294.0%4504205
colddifficulty 388.0%4503915
colddifficulty 477.1%10007715
cursorabsence-proof83.3%60503
cursoraggregation70.0%60423
cursorcalls97.5%2402343
cursorcollision61.7%60373
cursorcross-stack98.8%2402373
cursordb100.0%2402403
cursordispatch95.0%60573
cursorendpoints99.6%2702693
cursorfield-grain80.0%60483
cursorimpact97.2%1801753
cursormulti-hop98.3%60593
cursornameless-entry100.0%90903
cursorreality-check92.7%1501393
cursortypes98.0%1501473
cursordifficulty 1100.0%3003003
cursordifficulty 298.0%3002943
cursordifficulty 398.0%3002943
cursordifficulty 487.0%6005223

Every question carries one or more categories and a difficulty; each cell is the pass rate of an arm on the questions in that category (or at that difficulty), pooled over the selected models — by default the Anthropic models, which every arm ran. The number in each cell is the pass rate; the color is that cell's difference from bare Claude Code in the same column — blue above, orange below, neutral at the baseline — so a row reads as where an arm beats or trails the null hypothesis. Full figure, definition and disclosures →