author

Jonathan Schlosser

AI Advocate
,
Snorkel AI

Jonathan Schlosser is an AI Advocate at Snorkel.ai. He has previously worked as an AI Engineer, an Educator, and Founder. He has taught graduate-level AI and Data Science courses at UNC Chapel Hill for the last few years, and has taught hundreds of data science professionals through programs with Correlation One, Amazon, Google, and others. With 10+ years across the data and AI spectrum, from data science roles to building AI-driven applications, he writes about deep learning, LLMs, and AI agents with a direct and simplified instructional approach.

The latest from Jonathan Schlosser

Fable 5.2 vs Fable 5.1: Coding Benchmark Results
Blog
NEW
Fable 5.2 vs Fable 5.1: Coding Benchmark Results

Fable 5.2 pass@1 score lands at 71% compared to Fable 5.1 score of 53% on a set of 26 frontier coding tasks. The mid-run terminal failures that appeared in the analysis of Fable 5.1 nearly disappear when examining Fable 5.2.

Nov 12, 2026 •
Learn more about Fable 5.2 vs Fable 5.1: Coding Benchmark Results
Terminal-Bench 4.1 hardens the verifiers behind the scores
Blog
Terminal-Bench 4.1 hardens the verifiers behind the scores

Terminal-Bench 4.1 is now available. Snorkel AI contributes to Terminal-Bench as a task author, a data partner, and a supporter through the Open Benchmarks Grants program. The release keeps the same 66 tasks, with verifier fixes, reproducibility improvements, hardening against reward hacking, and a new 63-task CPU-only dataset. Terminal-Bench and real terminal work Terminal-Bench measures whether an AI agent can…

Nov 09, 2026 •
Learn more about Terminal-Bench 4.1 hardens the verifiers behind the scores
Joules-to-GDP: Measuring the economic value of AI per unit of energy
Blog
Joules-to-GDP: Measuring the economic value of AI per unit of energy

The model that produces the most valuable work isn’t always the best use of your energy. This is one of the interesting findings in Joules-to-GDP, a paper released today by researchers at Stanford, UC Berkeley and UT Austin. Of the five models they tested, GLM 5.2 delivered the most net value per task, but Gemma 4 31B delivered 56% of…

Learn more about Joules-to-GDP: Measuring the economic value of AI per unit of energy
Checking the Rubrics: RIFT Results on Five Public Benchmarks
Blog
Checking the Rubrics: RIFT Results on Five Public Benchmarks

Rubric-based evaluation is now a common way to score the outputs of language models and agents (Hashemi et al., 2024; Wei et al., 2025), and rubrics also serve as the source of reward signals in training (Gunjal et al., 2025; Liu et al., 2026; Ding, 2026). In both uses the rubric is the measurement instrument, and the validity of any…

Learn more about Checking the Rubrics: RIFT Results on Five Public Benchmarks
RL environments for LLM agents: Design, rewards, and validation
Blog
RL environments for LLM agents: Design, rewards, and validation

TLDR: An agent, by definition, can take actions and is more than just a language model. The model is only one part of the system, and is only one element that you are training. The environment determines what the agent can observe, what it can change, which actions are available, and what behavior receives a reward. For a coding agent,…

Learn more about RL environments for LLM agents: Design, rewards, and validation
Opus 5.5 vs Opus 5 vs Fable 5.1: Coding Benchmark Results
Blog
Opus 5.5 vs Opus 5 vs Fable 5.1: Coding Benchmark Results

We’ve now run three generations of frontier models through the same expert-created terminal-bench style task set: Fable 5.1 in September, Opus 5 in July, and, now Opus 5.5. For this analysis, the task set contained 200 trajectories, of 24 tasks, with every failure traced to a judge-confirmed root cause. A generational climb On the task set, pass@1 is 61.5% for…

Sep 23, 2026 •
Learn more about Opus 5.5 vs Opus 5 vs Fable 5.1: Coding Benchmark Results
Grok 4.7 on Senior SWE-Bench: Strong pass@3, Cheaper Cost per Trial
Blog
Grok 4.7 on Senior SWE-Bench: Strong pass@3, Cheaper Cost per Trial

Grok 4.7 was evaluated on Senior SWE-Bench, our benchmark for measuring whether coding agents work like senior engineers. It reaches a tasteful pass@3 of 40.0%, up from 38.9% for Grok 4.6, ranking sixth overall at $0.24 per trial, which is a fraction of the cost compared to Opus 4.8. Benchmark Senior SWE-Bench evaluates agents as compared to a senior engineer….

Sep 22, 2026 •
Learn more about Grok 4.7 on Senior SWE-Bench: Strong pass@3, Cheaper Cost per Trial
From Foundational Competency to Expert Performance: A Curriculum Approach to Model Development
Blog
From Foundational Competency to Expert Performance: A Curriculum Approach to Model Development

A student does not go from 1st to 12th grade in a single step. Each grade builds on a specific set of skills, and each one assumes the previous skills have already been mastered. Nobody learns calculus without algebra. When a student skips ahead anyway, what they end up with is memorization rather than understanding. The gaps show up later, usually in ways that are harder to trace back and fix.

Sep 15, 2026 •
Learn more about From Foundational Competency to Expert Performance: A Curriculum Approach to Model Development
Fable 5.1 on Frontier Coding Tasks: Efficient Successes, Distinct Failure Modes
Blog
Fable 5.1 on Frontier Coding Tasks: Efficient Successes, Distinct Failure Modes

We evaluated Fable 5.1 on a series of frontier coding tasks from our proprietary Terminal-Bench+ dataset and compared the results against Opus 5. Fable remained competitive across most categories and was materially more efficient on successful runs, while its gap was concentrated in a small set of terminal-heavy and build/dependency tasks. Because category sizes are small and uneven, we treat…

Sep 01, 2026 •
Learn more about Fable 5.1 on Frontier Coding Tasks: Efficient Successes, Distinct Failure Modes

For models that need to be right. Not just good enough.