From Foundational Competency to Expert Performance: A Curriculum Approach to Model Development

A student does not go from 1st to 12th grade in a single step. Each grade builds on a specific set of skills, and each one assumes the previous skills have already been mastered. Nobody learns calculus without algebra. When a student skips ahead anyway, what they end up with is memorization rather than understanding. The gaps show up later, usually in ways that are harder to trace back and fix.

Model training works the same way. Hard tasks are important, but their impact is limited when a model has not yet built the capabilities those tasks assume. A curriculum, meaning a deliberately ordered progression of tasks calibrated to what a model can currently do, gives research teams a way to locate where a model actually stands, identify the specific capabilities it is missing, and target the next round of data at exactly those gaps.

Curriculum learning as a training design decision

Curriculum learning is an established approach in the field. The idea that ordering training tasks from easier to harder can improve both convergence and final performance goes back to the early deep learning research literature: a learner extracts more signal from an example when it already has the prerequisites to interpret it.

For agentic coding tasks the argument sharpens into something specific. Consider a task that requires an agent to read a specification, implement a change across several files, run the test suite, interpret a failure, and revise. Now, consider that the agent fails because it never checks whether the output matches the specification. In a pass/fail evaluation, the signal the training process extracts from that failure would essentially be the same as the signal produced from a single-step task. Here, the multi-step, and potentially harder, task arrived at the same result, but did so in a more inefficient manner and wrapped in more complexity. The takeaway is that when an agent fails before reaching the relevant skill, a hard task may just provide a weak and expensive training signal for that skill and may just relocate the point of failure to someplace that is more expensive to observe.

The core claim is that the training value of a task is bounded by whether the model has the prerequisite capabilities to fail it for the right reason. A curriculum is the mechanism for making sure that condition holds, and it does so by making difficulty a measured property rather than an assumed one.

This also clarifies a distinction that is frequently blurred. A benchmark is an instrument. It is small, static, held out from training, and optimized for construct validity, meaning that a score reflects the capability it claims to measure. A training corpus is an intervention: it exists to change the model. It needs volume, coverage across the capability space, and a reward structure capable of shaping behavior. The two can share a task format and a verification philosophy, but they answer different questions, and a curriculum belongs firmly on the intervention side.

What a curriculum for coding agents actually requires

Translating the school analogy into a training corpus imposes some concrete requirements. They are worth stating up front because they become the lens for evaluating whether a given dataset functions as a curriculum or is simply a large collection of tasks.

1. Difficulty must be measured, not authored. A task author’s estimate of difficulty and a model’s measured pass rate commonly disagree enough that a curriculum built on authored intent will have rungs in the wrong place. Difficulty must be assigned empirically, against the models the curriculum is meant to train, and re-assigned as those models improve.
2. Coverage must be broad enough that failure modes separate. If every task shares the same language, codebase scale, and specification style, a failure cannot be attributed to a skill rather than to a surface feature. Diversity across task type, environment, and prompt is what gives failure data enough contrast to be diagnostic.
3. Reward must be dense enough to localize failure. A single terminal pass/fail signal says a task failed, not where. For multi-step work, reward has to land at intermediate points so that the training process can distinguish an agent that failed at step one from an agent that failed at step five.
4. Verification must be resistant to reward hacking. If a task can be passed by producing something that looks right without being right, the curriculum will teach exactly that. Deterministic checks are the strongest defense where the output allows them; where it does not, as with LLM-as-judge rubrics, the judge itself has to be validated against the same risk. 

These properties can distinguish a training curriculum from a benchmark that has been scaled up.

Terminal-Bench

Terminal-Bench is one of the most widely referenced benchmarks for coding agents. It places an agent in a live container with a terminal, an instruction, and whatever assets the task requires. The agent acts until it declares completion and a verifier inspects the final environment state. Grading is a mechanical process and is not necessarily judged.

Its own evolution is an illustration of a curriculum being assembled in public, one tier at a time, as frontier models advance their capabilities. 

Training only on the newest benchmark is a concentration risk. It focuses data collection on the currently visible frontier of difficulty while potentially underrepresenting prerequisite behaviors, diagnostic contrast, and reward-localization. Tasks from earlier benchmark versions can provide controlled coverage of the skills that harder tasks test. 

Each version of Terminal-Bench offers value; each one provides differences in capability and sets of skills. But a benchmark, by design, cannot be a curriculum. It is too small to train on, it must stay static to remain comparable, and a single pass/fail per task cannot localize where a failure occurred.

Terminal-Bench+ as a curriculum in practice

Terminal-Bench+ extends Terminal-Bench into a training-scale data series organized as a progressive curriculum. It contains more than 50,000 expert-authored tasks across 12 task types and 9 languages, with codebases as large as 200+ files. Every task ships Harbor-native with deterministic tests, rubrics, and filterable metadata.

The curriculum includes a diversity of tasks that span as a deliberate gradient. At the foundational end are atomic tasks: a single skill exercised in a small or empty codebase, Python-heavy, with short context and end-state grading. In the middle are multi-step, longer context work in codebases with dozens of files and multi-container environments. At the expert end are long-horizon workflows in large repositories.

Each expert-authored task is calibrated for difficulty against current frontier models and goes through an extensive quality control review process. The training curriculum is designed to surface failure modes that can provide signals for targeted training.  

What the curriculum surfaces

We conducted a controlled analysis over a sample of Terminal-Bench+ tasks with GPT-5.5 and Claude Opus 4.8: one standardized scaffold, five rollouts per task, 740 trajectories, with every failing trajectory coded against a fixed taxonomy defined in advance. The findings demonstrate what a training curriculum makes visible.

Foundational tier: 

Skipping Verification 

The dominant failure mode at 43% of all failures within the analysis, was faulty inference. The agent runs a sequence of commands, none returns an explicit error, and it concludes the task is complete without checking the output against the specification, even when the check was one command away. For example, in a task to build a security reconciler, the agent failed to identify a boundary edge case. It didn’t run the test suite before marking it complete, so it ended up missing the boundary bugs. 

It is the top failure mode for both models despite a six-point gap in pass rate, which points to a cross-model bottleneck. And it is a foundational habit that every higher rung assumes; a model without it fails expert tasks for reasons unrelated to expert skill.

It is also silent and critical, which is why scale alone has not fixed it: under sparse end-state reward mechanisms, an agent that verifies and one that happens to be correct can receive the same outcome. That can under-credit verification and weaken the incentive to learn the behavior consistently. Milestone grading and process rubrics work to break that condition.

Intermediate tier: 

Misreading requirements and following the wrong strategy

Misreading requirements occurs at planning time, so every subsequent action inherits it. An example of this is a debugging task where the agent attempted to patch the Go simulator using here-docs, but literal tabs in the input triggered bash tab-completion, corrupting the source files and causing the build to fail. 

Failing to revise a plan after an error cascades: the agent sees the fault and keeps executing around it. Both are comprehension problems pointing toward mid-training data, and both only become observable under production-realistic conditions, where milestone grading turns “failed the task” into “succeeded at milestones 1 and 2, failed at 3.”

After already observing truncation, the agent retried by requesting large multi-section documentation output in one shell execution, failing to address the output-limit problem that later consumed the run.

Behavior under reward: poor efficiency / unnecessary rework

Redundant exploration, re-reading files and re-running commands whose output is already available, is a behavior-under-reward problem that localizes to post-training and RL. The series’ agent-rollout diversity controls exist so that this behavior is exercised rather than left to emerge.

Expert tier: 

Navigating complex scenarios and deliberate decoys

As the skills expectations get harder, so do the assessments. For example, in a Rust debugging task where a driver must rebuild some billing processes with an expected output, the agent is inconsistent across handoffs, tombstone rows, digests, and audit paths. Every contract is documented in the environment but the hand-authored output is rejected. This resulted in only 10% of attempts passing, with faulty inference in seven of the nine failures.

Another example of an expert-level difficulty task is a security audit that assigns one of twelve verdicts under a security contract spanning multiple specification files, from key ordering to a self-binding SHA-256 digest. Two input fields in this are deliberate decoys. Only 20% of attempts passed and recognized those decoys.

The decoys are the point though. They exist because the author anticipated the behavior of an agent reviewing the data records: it would read enough of the specification to begin, then proceed on a partial understanding, and would ultimately go wrong. PR-mined tasks can provide valuable realism, but they do not necessarily guarantee deliberate coverage of anticipated agent failure modes. Expert-authored tasks can complement them by testing specific hypotheses about where coding agents go wrong.

How a curriculum-based approach advances capability

The failure analysis explains why a curriculum is diagnostic. But the approach also builds capability more effectively than an undifferentiated pool of hard tasks.

1. Signal efficiency. A learner gets the most usable gradient from tasks whose rollouts vary: some attempts land close, some are far off, and some sit in between, which gives the reward something to discriminate between. Tasks the model always solves or always fails produce near-identical rollouts and almost no signal. A calibrated gradient keeps more training compute on the tasks where that variance is highest.
2. Progressive skills transference. Skills generally transfer up the difficulty ramp, not downwards. A curriculum makes it possible to test which skills generalize across task complexity and which require targeted training. An agent that has learned to verify its output carries that habit into every longer task; an agent that has memorized one frontier task has learned nothing that helps on the next. This is why the order of a curriculum matters, not just its coverage.
3. Closing the loop between evaluation and training. When the corpus is calibrated, tagged, and milestone-graded, its failure distribution is the training plan: faulty inference points at atomic tasks with process rubrics, breaking at the third milestone points at production-realistic tasks at that depth. Moving from diagnosis to data is a filter over one corpus, not a new collection effort.
4. Diversity. The variation within a skill turns pattern-matching into competence. A model that sees the same skill across nine languages, three codebase scales, and six prompt axes is forced to learn the invariant, because nothing else predicts reward.
5. Difficulty calibration. A curriculum is a moving target by construction. As a model improves, re-measuring difficulty keeps the informative band aligned with the learner.

Taken together, these are why the ceiling for improvement is not fixed by scale. Difficulty is not the training signal; the located gap is, and closing those gaps in order builds competency rather than the appearance of it.

The takeaway

Curriculum learning turns a training corpus from a pile of hard tasks into a diagnostic tool and a plan. It requires difficulty that is measured rather than assumed, coverage broad enough that failures separate by skill rather than by surface form, reward dense enough to say where a failure occurred, and verification that cannot be satisfied by output that merely looks finished. Terminal-Bench+ is built to those requirements as a single series spanning atomic terminal skills through frontier-difficulty engineering work, and when frontier models are run against it the result is not a score but a map: verification habits missing at the foundation, composition breaking down in the middle, and dense interlocking constraints defining the ceiling. Each of those is a specific, addressable gap with a specific place in the training pipeline, and closing them in order is how a model moves from basic competence to expert performance.

Interested in learning more or requesting Terminal-Bench+ data samples? Request samples.