Figure 16 · Epoch-4

Organic index adoption by model

Every arm here was given a codebase index and left to decide whether to use it: the prompt named no tool, each indexing tool was installed as shipped with its own instructions in place, and every tool was directly visible to the model. This figure is how often the harness actually called the index it had, per model.

protocols
battery

Every model is on the horizontal axis. Protocols pool with equal weight per protocol × battery cell; pick one protocol to read it alone. MultiTurn is one conversation per battery, so its per-question share is lower by construction; the table carries the per-conversation figure.

Organic index adoption by modelshare of question runs · not a scoreZeroShot + MultiTurn · both batteries
0%25%50%75%100%Haiku 4.5Sonnet 5Opus 4.6Opus 4.8Opus 5Fable 5GPT 5.6 LunaGPT 5.6 TerraGPT 5.6 SolGrok 4.6Composer 2.5MODEL · FAST → FRONTIERCorograph 0%CC Bare 0%Codex Bare 0%Cursor Bare 0%Graphify 33%GitNexus 12%CodeGraph 25%CodebaseMemory 7%Serena 0%
Data behind this figure — ZeroShot + MultiTurn · both batteries
armrolemodelshare of question runs
calling the index		question runswith an index callindex callssetup calls
(excluded)		no tool callsconversationsconversations
with an index call		runs in the repository
CorographCorographHaiku 4.50%98000—980390Haiku 4.5
Sonnet 50%100000—1000400Sonnet 5
Opus 4.60%100000—1000400Opus 4.6
Opus 4.80%100000—1000400Opus 4.8
Opus 50%100000—1000400Opus 5
Fable 50%33000—330110Fable 5
GPT 5.6 Luna0%100000—1000400GPT 5.6 Luna
GPT 5.6 Terra0%100000—1000400GPT 5.6 Terra
GPT 5.6 Sol0%100000—1000400GPT 5.6 Sol
CC Barebare harnessHaiku 4.50%90000—37360Haiku 4.5
Sonnet 50%100000—193400Sonnet 5
Opus 4.60%100000—60400Opus 4.6
Opus 4.80%70000—28300Opus 4.8
Opus 50%100000—3400Opus 5
Fable 50%40000—19150Fable 5
Codex Barebare harnessGPT 5.6 Luna0%100000—3400GPT 5.6 Luna
GPT 5.6 Terra0%100000—8400GPT 5.6 Terra
GPT 5.6 Sol0%100000—9400GPT 5.6 Sol
Cursor Barebare harnessHaiku 4.50%100000—13400Haiku 4.5
Sonnet 50%100000—15400Sonnet 5
Opus 50%100000——400Opus 5
GPT 5.6 Luna0%100000—60400GPT 5.6 Luna
GPT 5.6 Terra0%100000—47400GPT 5.6 Terra
GPT 5.6 Sol0%100000—73400GPT 5.6 Sol
Grok 4.60%100000—1400Grok 4.6
Composer 2.50%100000——400Composer 2.5
Graphifyindexing tool on Claude CodeHaiku 4.52%9002222—763616Haiku 4.5
Sonnet 523%1000239239—424020Sonnet 5
Opus 4.60%100000—69400Opus 4.6
Opus 4.814%7008787—193014Opus 4.8
Opus 53%10003030—24016Opus 5
Fable 533%400142142—141512Fable 5
GitNexusindexing tool on Claude CodeHaiku 4.542%9003881342—263633Haiku 4.5
Sonnet 534%1000322719—194038Sonnet 5
Opus 4.636%1000344600—584030Opus 4.6
Opus 4.814%70081227—203020Opus 4.8
Opus 520%1000200319——4033Opus 5
Fable 512%4005177—141513Fable 5
CodeGraphindexing tool on Claude CodeHaiku 4.568%9506541403—633838Haiku 4.5
Sonnet 549%1000464860—494031Sonnet 5
Opus 4.636%1000343524—1524030Opus 4.6
Opus 4.837%700224303—333023Opus 4.8
Opus 534%1000333378—34027Opus 5
Fable 525%400103131—221510Fable 5
CodebaseMemoryindexing tool on Claude CodeHaiku 4.580%9007124379810143636Haiku 4.5
Sonnet 57%100059312194264027Sonnet 5
Opus 4.614%1000146317252684022Opus 4.6
Opus 4.84%70027148232203023Opus 4.8
Opus 52%1000176332324016Opus 5
Fable 57%400296814618159Fable 5
Serenaindexing tool on Claude CodeHaiku 4.57%9506471262153821Haiku 4.5
Sonnet 50%100011129401Sonnet 5
Opus 4.60%1000139763401Opus 4.6
Opus 4.80%70011134301Opus 4.8
Opus 50%10001211401Opus 5
Fable 50%48000—14190Fable 5

Figure 16. Every arm here was given a codebase index and left to decide whether to use it: the prompt named no tool, each indexing tool was installed as shipped with its own instructions in place, and every tool was directly visible to the model. This figure is how often the harness actually called the index it had, per model. Each line is one arm across the models it ran, from the fastest to the frontier model; a point is the share of that arm's question runs at that model in which at least one tool call went to the arm's index, with the tool's setup calls excluded. Hover a point for the counts; the table beneath carries every number. Read it as a description of behaviour rather than a ranking: it says how much of each arm's result was produced with its index in the loop, and how that changed as the model got stronger.

How this figure is computed

Definition. A question run is one question answered by one arm at one model in one repetition (a ZeroShot session, or one turn of a MultiTurn conversation). It counts as using the index when at least one of its main-thread tool calls is an index call: for Corograph, a query to its codebase index; for an indexing tool served over MCP, a call to that tool's own tool surface; for Graphify, which ships as a skill rather than an MCP server, an invocation of that skill. Calls that prepare a tool rather than ask it a question — project activation, instruction fetches, index build or status checks, schema reads — are excluded from the count and listed per arm in the summary file under setup_tools_excluded, so a reader can put them back.

Aggregation. For an arm at one model, the share is computed within each protocol × battery cell and the cells are averaged with equal weight, the same pooling law as every other figure, so the longer battery and the larger protocol never outvote the others. The table beneath the figure reports the summed counts as well as the share.

Setup. The per-question prompt is published in the repository and names no tool. Each indexing tool was installed by its own installer into the container's home directory, which is where those installers write their MCP registration, instructions, hooks or skill; GitNexus and CodeGraph also built their index before the first question, as their documentation directs, and GitNexus wrote its own context files into the working copy. Claude Code's deferred tool loading was switched off for every arm, so the full MCP tool surface was visible to the model without a search step. Corograph ran through its own gateway with its index available from the first call. Nothing on either side prompted the model toward or away from any tool.

Reading it. The vertical axis is the share of question runs that used the index; it is not a score. A low value means the harness answered mostly by reading and searching the source with the index unused; a high value means the index was in the loop for most answers. The figure should be read with Figures 1, 2 and 8: an arm's accuracy and token results were produced with its index used this often.

Disclosures

Main thread only: MCP tools are not offered to sub-agent threads, so a sub-agent's reads and searches do not enter this figure. HumanExam is whole-battery and has no per-question tool attribution; it is not shown.

MultiTurn is reported per question; one conversation carries a whole battery, so a conversation that consulted the index once early is counted at the questions where it did so. The table reports the per-conversation figure beside it (conversations with at least one index call).

Graphify ships as a skill: its graph was built before the session and the skill invocation is the index call; reads of the graph files by other means are not distinguishable from ordinary file reads and are not counted.

CodebaseMemory indexes on demand: its index was not built before the session, so its index_repository and index_status calls are setup calls, excluded here and listed in the summary file. Serena has no prebuilt index by design; its project activation and instruction calls are setup calls likewise.

The share says how often the index was called, not whether its answer was used: a run that queried the index and then read the source is counted the same as one that answered from the query. The tool sequences themselves are in each run's transcript.

Provenance

Computed at build time from epochs/E004/summary/adoption.json of the proof package (export tool tabench export public 0.1.29, scorer era tabench-1.1). Each row of the table above links to the run directories in the repository, where run.json, the answers, the verdicts and the transcript of every run can be read.