Research

Training agents for professional work when success is rare

November 19, 2026
•
9 min read
•

We trained Qwen3.8-27B on 1,105 professional tasks from Snorkel’s GDPval+ series. After about 102K training rollouts, the model reaches 40.7% pass@1 on OpenAI’s GDPval gold set, up from 31.9%. Its deliverables now beat those of human experts in 44.0% of LLM-judged head-to-head comparisons, compared with 36.4% for the base model. The gains come from two ingredients working together, expert-built training data and binary-gated GRPO, the algorithm we introduced in our previous post. On the same data and with a similar number of rollouts, standard GRPO with a binary signal barely moves off the base model.

020K40K60K80K100KTraining rollouts generated26%28%30%32%34%36%38%40%42% baseGDPval pass@1 (rubric ≥ 90%)

Figure 1. Pass@1 on 185 GDPval tasks, averaged over eight attempts per task. Dots are individual evaluations and lines are smoothed trends. Each method was trained once, and the labeled point is the best binary-gated evaluation within 102K rollouts.

More and more, we ask AI agents not to answer questions about professional work but to do it. OpenAI’s GDPval measures exactly that, with tasks drawn from experienced professionals across nine sectors of the US economy. Each task ends in a real deliverable, such as a financial model, a regulatory filing, or a slide deck built from a folder of source documents. Every task comes with a reference deliverable produced by a human expert and a detailed rubric listing what a complete deliverable must contain. These are long, demanding jobs, and in our evaluations a single attempt averaged about 50,000 response tokens and could involve numerous tool calls.

Our training data comes from Snorkel’s GDPval+, a series of thousands of professional tasks built by domain experts, each with its own detailed rubric and reference solution. On GDPval, we count an attempt as a pass when it satisfies at least 90% of its rubric items. We use 90% rather than 100% because LLM judges are noisy, and grading the same deliverable twice can flip the verdict on an individual item.

On the training side, we use binary-gated GRPO, which we introduced in our previous post. Binary-gated GRPO works with any score that grades a whole attempt, and on GDPval+ that score combines two judgments. A judge first measures the weighted share of rubric items that the deliverable satisfies. That share is then multiplied by a factor between one half and one, set by whether the deliverable’s key results and conclusions agree with the expert’s reference solution. Unlike the evaluation pass, a training attempt passes when this combined score reaches 90%. The previous post showed that such a score helps when it carries information that a pass or fail label discards and when improving it moves the model toward success. The reference factor targets the second condition directly, since a deliverable that satisfies many rubric items but gets its key figures wrong loses up to half of its score.

Binary-gated GRPO combines the two signals with a strict priority. It learns from pass or fail whenever a group contains a pass, and turns to the training score only when every attempt fails, so a failed attempt is never reinforced alongside a successful one. GRPO, by contrast, sees only whether each attempt passed, and on tasks this hard most groups contain no pass at all. Trained on the same data, it stays close to the base model, while binary-gated GRPO lifts pass@1 from 31.9% to 40.7%. In the rest of this post, we show why GRPO stalls on these tasks, how binary-gated GRPO recovers the signal it misses, and how the gains hold across metrics, sectors, and response budgets.

Why GRPO loses signal

GRPO samples a group of rollouts for each task, eight in our runs, and scores each rollout by the difference between its return and the group mean. Rollouts above the mean are reinforced and those below it are discouraged, so several rollouts in one group can be reinforced together. With a binary return of one for a pass and zero otherwise, this comparison is sharp whenever a group mixes passes and failures. When every rollout fails, all eight returns are zero and the group is silent.

On tasks this hard, silence is the common case rather than the exception. Both of our runs use dynamic sampling, a common practice that skips silent groups and refills the batch with fresh tasks, so that each update is built only from groups with signal. It changes how the model is updated, but it does not create any additional learning signal, since a discarded group would have contributed nothing anyway. Under GRPO, 79% of all generated groups contained no passing rollout, so four in five groups contributed nothing to training.

Binary-gated GRPO

Those discarded groups were far from uniformly poor. In the median discarded group, the best rollout reached a training score of 78%, and 66% of discarded groups contained a rollout that scored at least 70%. A binary return scores all of these attempts as zero, so the model never learns which failures came closer to a finished deliverable.

The training score carries exactly this information, but it cannot simply take the place of the pass criterion. In the previous post, we showed formally and through a concrete example that maximizing such a score can diverge from completing tasks. A policy can collect more partial credit while finishing fewer of them. The score is therefore most valuable as a source of information when the binary return has nothing to say.

Binary-gated GRPO is built on this principle, so any group that contains at least one pass receives exactly the advantages it would receive under GRPO. Only when every rollout in the group fails does a fallback based on the training score take over.

Ai = (Bi − B̄) + 𝟙[no pass] · η (Di − D̄),
Bi = 𝟙[Ri ≥ 0.9] (1 − Wi),   Di = clamp((Ri − 0.4) / (0.9 − 0.4), 0, 1)

Here Ri is the training score of rollout i, and Wi marks a rollout that exhausted its training budget, so Bi is the binary return. The fallback Di sends every score at or below 40% to zero, so a weak deliverable earns no more than no deliverable at all. Above that floor, it stretches scores linearly up to the 90% pass threshold, and η = 0.85 sets its weight.

90% pass threshold28%0−0.3546%0−0.2547%0−0.2352%0−0.1571%0+0.1774%0+0.2477%0+0.2878%0+0.30GRPObinary-gatedtrainingscoreGRPO21 of 100 groups retainedOne discarded groupEight rollouts and their advantagesBinary-gated GRPO95 of 100 groups retainedretained · passretained · fallbackdiscarded · zero advantage

Figure 2. Each square is 1% of the groups generated in the first 102K rollouts. The middle panel expands one group that GRPO discarded.

With the gate in place, 95% of generated groups carried a training signal, compared with 21% under GRPO. Groups with no pass made up 67% of all generated groups and carried 55% of the absolute advantage mass. The fallback supplied most of the signal in precisely the regime where GRPO had none.

On the training tasks themselves, binary-gated GRPO improves far more quickly across the board. GRPO still learns, but only from the minority of groups that contain a pass.

025K50K75K100KTraining rollouts generated0.40.50.60.7Mean training score025K50K75K100KTraining rollouts generated5%8%10%12%15%Training score ≥ 90%025K50K75K100KTraining rollouts generated0%20%40%60%80%100%Groups retainedBinary-gated GRPOGRPO

Figure 3. Metrics cover all generated groups, including those later discarded by dynamic sampling. Dots average 2,048 rollouts.

Where the gains show up

A rubric checks whether a deliverable contains what it should. GDPval’s expert comparison asks a different question, namely whether the deliverable is better than the one a human professional produced for the same task. On that measure, judged by an LLM following the benchmark’s protocol, the binary-gated model’s work was rated above the expert’s in 44.0% of attempts, up from 36.4% for the base model. Counting ties, as GDPval’s headline metric does, the trained model matched or beat the expert in 47.6% of attempts.

The same pattern holds under every measure we tracked. Binary-gated GRPO rises above the base model at both rubric thresholds and in expert comparison, both for pass@1 and for pass@8, which counts a task as solved if any of its eight attempts succeeds. GRPO hovers around the base model on most of them.

28%32%36%40%pass@1 baseRubric ≥ 90%14%16%18%20%22% baseRubric ≥ 95%32%36%40%44% baseExpert comparison050K100KTraining rollouts generated52%56%60%64%pass@8 base050K100KTraining rollouts generated32%36%40%44% base050K100KTraining rollouts generated52%56%60%64% baseBinary-gated GRPOGRPO

Figure 4. Dots are individual evaluations and lines are smoothed trends. Dashed lines mark the base model.

The improvement is also broad, with binary-gated GRPO ahead of the base model in all nine GDPval sectors on rubric pass and in eight of nine on expert comparison.

30%40%50%Information (14)Real Estate and Rental andLeasing (20)Professional, Scientific, andTechnical Services (24)Manufacturing (25)Health Care and SocialAssistance (22)Retail Trade (16)Wholesale Trade (25)Government (22)Finance and Insurance (17)Rubric pass@1 (≥ 90%)20%30%40%50%Expert comparison, pass@1Base modelBinary-gated GRPO

Figure 5. Binary-gated values average three evaluations, and base-model values come from a single evaluation.

The trained model also runs out of budget far less often. The share of attempts that ran out of response budget fell from 17.2% to 7.0%, while under GRPO it stayed above 11% in every evaluation. Unlike GRPO, the fallback adds a training signal to many groups where every attempt fails. Within these groups, attempts that finish typically score above those cut off by the budget.

This improved ability to finish is reflected in performance under tighter token budgets. Capped at 64K tokens, about half the full budget, the trained model already matches the base model’s full-budget pass rate.

32K48K64K80K96K128KResponse-token cap15%20%25%30%35%40%reaches the base-modellevel by 64KPass rate within shorter response budgets

Figure 6. An attempt counts only if its original response ends within the cap. Responses are not truncated or rescored.

Closing thought

When success is rare, most of what a model can learn lies in how its failed attempts differ. Binary-gated GRPO puts that variation to work without letting it redefine success, and expert-built tasks with detailed rubrics and reference solutions make the variation worth learning from. Together, they took a 27B model from 31.9% to 40.7% pass@1 on GDPval, with its deliverables rated above the expert’s in 44.0% of head-to-head comparisons.

Share this article
Image
Mehdi Fatemi
Principal Research Scientist

Mehdi Fatemi is a Principal Research Scientist at Snorkel AI and a reinforcement learning (RL) researcher with more than 15 years of experience, beginning with his PhD at McMaster University. His work spans the foundations and applications of RL, including value-based methods, long-horizon and multi-turn decision-making, and, over the past few years, reasoning in large language models under diverse feedback signals and uncertainty. Previously, during several years at Microsoft Research, he made numerous core contributions to RL through papers, patents, and public releases, including Hybrid Reward Architecture for learning from decomposable rewards and dead-end theory for identifying and avoiding irrecoverable states. His recent work includes Concise Reasoning, which shows that RL’s loss alone drives reasoning models toward verbosity or conciseness, depending on the solvability of the training tasks.

Recommended articles

View all articles
Anthropic Fable 5.2 cover image
Fable 5.2 vs Fable 5.1: Coding Benchmark Results
Fable 5.2 pass@1 score lands at 71% compared to Fable 5.1 score of 53% on a set of 26 frontier coding tasks. The mid-run terminal failures that appeared in the analysis of Fable 5.1 nearly disappear when examining Fable 5.2.
November 12, 2026
•
Ankit Aich
,
Jonathan Schlosser
Image
Terminal-Bench 4.1 hardens the verifiers behind the scores
Terminal-Bench 4.1 is now available. Snorkel AI contributes to Terminal-Bench as a task author, a data partner, and a supporter through the Open Benchmarks Grants program. The release keeps the same 66 tasks, with verifier fixes, reproducibility improvements, hardening against reward hacking, and a new 63-task CPU-only dataset. Terminal-Bench and real terminal work Terminal-Bench measures whether an AI agent can
November 9, 2026
•
Jonathan Schlosser
Snorkel AI Reading Group, San Francisco: Intelligence per Watt with speakers Jon Saad-Falcon and Avanika Narayan
Intelligence per Watt: Measuring Local AI Efficiency
Jon Saad-Falcon and Avanika Narayan (Stanford, Hazy Research) present Intelligence per Watt, a metric that divides task accuracy by power draw to compare local AI models and hardware, and show it improved 5.3x from 2023 to 2025.
November 5, 2026
•
Snorkel Team
Image

Join our newsletter

For expert advice, the latest research, and exclusive events.
By submitting this form, I acknowledge I will receive email updates from Snorkel AI, and I agree to the Terms of Use and acknowledge that my information will be used in accordance with the Privacy Policy.