Opus 5.5 vs Opus 5 vs Fable 5.1: Coding Benchmark Results

We’ve now run three generations of frontier models through the same expert-created terminal-bench style task set: Fable 5.1 in September, Opus 5 in July, and, now Opus 5.5. For this analysis, the task set contained 200 trajectories, of 24 tasks, with every failure traced to a judge-confirmed root cause.

A generational climb

On the task set, pass@1 is 61.5% for Fable 5.1, 60.7% for Opus 5, and 60.7% for Opus 5.5. This is a flattening out of this score across the last two generations, with Fable 5.1 nominally higher than the other two. Pass@5 has Fable 5.1 scoring 74.1%, Opus 5 scoring 79.3%, and Opus 5.5 scoring 76.7%. Here, Opus 5.5 actually falls short of Opus 5. But, if we look at all of the trajectories taken as a whole, Fable 5.1 passed 49% (94/191), Opus 5 passed 61% (118/195), and Opus 5.5 passed 68% (136/200). If we exclude the runs where the model had difficulty in receiving the correct format, this value goes up to 74% (184/200). This overall pass rate shows improvement model over model for the task sets and continued growth, with Opus 5.5 at the top.

Three models, three failure signatures

The most useful output of trajectory-level analysis isn’t the pass rate but rather insights on how each model fails. On our task set the three generations fail in different ways.

Fable 5.1 struggled mid-run. Its failures were dominated by premature termination and unrecovered errors with reasoning errors playing a minor role.

Opus 5 reasoned its way into failures. Faulty inference alone accounted for 35% of its judge-confirmed root causes, the largest single mechanism by a wide margin, while operational failures (e.g. tool use, execution, termination) were rare. Its ceiling was the model’s conclusions, not its mechanics.

Opus 5.5 fails at the edges. Its top root causes were:

• Wrong output format — 25% of failures. Every one of these occurred at the start of the run: an empty or non-structured first response the harness could not parse, so no commands were ever executed. Notably, these concentrate almost entirely in two tasks — one of which also fails at the harness level for Opus 5 on every attempt — pointing to a model–harness interface problem rather than an engineering deficit.
• Faulty inference — 20% of failures. This is down by nearly half from Opus 5’s 35%. The pattern here is that the model commits to a wrong conclusion and that leads it astray. In one example, the model though that standard inclusive day-counting in a billing calculation was a bug and then focused on “fixing” correct code.
• Wrong tool arguments — 19% of failures. The model would pick the right tool, but it would pass the wrong parameters. Although, this was clustered heavily on a small number of tool-intensive tasks rather than spread across the set.
• Partial execution — 12%. The model completes some of the required work and stops short of the rest.

The premature termination that defined Fable barely registers, and the reasoning ceiling that defined Opus 5 has visibly receded. Since the format failures never engage the model’s engineering ability at all, a capability-adjusted view that excludes them puts Opus 5.5’s pass rate at roughly 74%.

When Opus 5.5 fails, it fails by compounding

Failing trajectories averaged 2.5 confirmed errors each with only a handful of tasks that failed on a single mistake. This reveals that the typical failing run is a cascade and not just a one-off error. Here an early inference or tool error occurs, then it propagates, with weak verification appearing repeatedly throughout the task. The agent essentially treats a command that ran without error as evidence the output is correct, letting the original mistake travel all the way to the final answer without actually correcting it, or even noticing it.

The cleanest discriminator between passing and failing runs is omission though and not quality. Skills rated “missed” appear in 1% of passing traces versus 9% of failing ones while “strong” ratings drop only modestly (88% to 70%). Opus 5.5 rarely does the work badly but when it fails, it is more often because a required step was never attempted at all.

Opus 5.5 also doesn’t fail randomly across the task set. It fails on the same handful of tasks repeatedly and extra attempts don’t really help. This can be seen with pass@5 (task-level, strict) landing at 76.7% and not climbing much further beyond that. The tasks it never solves are ones that are genuinely hard for every model we’ve tested — Opus 5 goes 0-for-5 on them too, just like Opus 5.5 does.

The takeaway

Across three generations on one task set, the story is steady gains with some moving challenges. Fable 5.1 struggled with terminal interactions, Opus 5 presented a reasoning ceiling, and now, Opus 5.5, has a failure profile primarily dominated by things adjacent to the model. this includes things like a protocol handshake that sometimes never happens, tool arguments on a handful of tasks, or faulty verification that lets early mistakes compound. These are more actionable than a knowledge ceiling.