Continual Learning Bench: Evaluating Whether AI Systems Learn

At our latest Snorkel AI Reading Group, Parth Asawa (UC Berkeley Sky Computing Lab) presented Continual Learning Bench, the first expert-validated benchmark designed to measure whether LLM-based systems genuinely improve through sequential experience. CL-Bench spans six domains, software engineering, signal processing, disease outbreak forecasting, database querying, strategic game-playing, and demand forecasting, and every task sequence is built around a latent structure a stateful system can discover online but a stateless one cannot. The question it asks is simple and almost never asked: when a model reaches the tenth problem, did any of the first nine help?

Transcript

Lightly edited for readability.

We measure what models can do, not how much they learn

My name is Parth Asawa. I’m a PhD student at Berkeley working on continual learning, and I’m going to talk about a work we did a few months ago on evaluating it.

We’re all at conferences and events like this one quite often, talking about what these models are doing. They’re learning, they’re really good at these various capabilities. But something you might notice is that we don’t actually talk about how much they learn.

If you’re familiar with how we measure models today, you’ve seen the graphs. Every time a new model is released we get a wide variety of single numbers on a lot of different benchmarks, telling us what percentage of problems the model solved. The way we get those numbers is that we ask the model to do one problem, say from Terminal-Bench, then independently another, then independently another, and we repeat that across a wide variety of benchmarks. We take individual questions and pretend the model has no memory, that it’s amnesic, that every time it does a new thing you’re starting it from scratch. Because all the instances are independent, we can collapse the evaluations into a single number. It’s the i.i.d. assumption that lets us do that.

Breaking a capability down like this is a bit like watching a human do one thing and then another, and pretending that in between the two they forgot everything about the first. The evaluation looks like a scatter around a mean. That’s not what learning should look like. Learning should look like improvement over time, and arguably continual learning is just learning. It should look like a curve where performance improves as a function of experience. That’s not captured in evaluations today.

What continual learning actually means

I define continual learning as sample-efficient online learning that’s stable over long horizons.

Think about how we train language models. We get training data, come up with algorithms and loss functions, and put the model in a box for weeks or months. After that offline process we take a frozen checkpoint out and deploy it. Those weights don’t change. Any adaptation afterwards happens in context, so these models are frozen in a snapshot of time. Continual learning would look like models that continuously adapt, take in new information, update their representations, and improve over time.

In language models there are roughly three ways people have tried this. There’s the context itself, since in-context learning is an emergent capability where the model adapts based on whatever is in the context. There’s externalized memory, where models write to notepads, structured memory stores, or vector databases, so you can move information out of the context and selectively choose what to include. And there’s parametric learning. I’m not going to argue for one right now, though I have strong opinions I’ll share later. What I will say is that in any of these cases we’re not evaluating continual learning in the way we claim to.

I can probably bet you that if you took any three papers from the last few years claiming to do continual learning and looked at their evaluation methodologies, almost all of them would be completely different. That’s not how machine learning makes progress. Typically we know what we’re optimizing for. We come together as a community and define criteria for a capability we want. We don’t have that for continual learning, which may be why the field has progressed slower.

Continual learning is not a new concept. It’s been in neural networks for decades, and it was popular in the mid to late 2010s. Evals looked like this: train on one subset representing one distribution, train on a second representing a different one, then evaluate on prior domains as well as new ones. If the model doesn’t regress on the prior domains, we call that continual learning. There’s also a class of recall or memory-based evaluations that use very long contexts to test factual recall.

None of these are actually measuring continual learning. Some aren’t testing learning at all. Some don’t measure sample efficiency, which is a first-order requirement. And some are in settings that are just incredibly unrealistic compared to the real world.

Why you can’t just chain existing benchmarks

The question I got most often while working on this was: why can’t I just chain together existing benchmarks? Take AIME 2025, have the model do problem one, then two, then three, and evaluate the sequence. Doesn’t that obviate the issues you’re describing?

You can’t, because benchmark instances for language models are fundamentally designed to be independent. That’s the i.i.d. assumption again, and it’s exactly what breaks a chained evaluation. There is no shared structure across AIME problems. I’ve taken some of these exams. If you do problem one, you probably won’t do much better at problem two.

A continual learning task has to be designed so that earlier experience is expected to lead to improvement. We want to break the i.i.d. assumption and create a mechanism that actually allows for learning.

Three design criteria

There are essentially three design criteria that go into building a continual learning benchmark.

The first is headroom. If I evaluate GPT-5 on my task and later evaluate GPT-6, and neither exhibits a different way of doing continual learning, I shouldn’t expect the stronger general model to improve on my benchmark. I’m not testing raw capabilities. I’m testing the ability to learn.

The second is shared structure. In continual learning you want some shared latent structure between tasks that an optimal learner can discover and exploit over time. If that structure doesn’t exist, it’s really hard to make any claims, because you might just not expect the model to improve at all.

The third is a learning mechanism. Whatever the environment, there has to be feedback available: the right answer, the right solution, compiler error messages, human feedback for a personalization task. If a feedback mechanism doesn’t exist, it’s hard to expect the model to be continually learning.

The gain metric

If we’re no longer looking at one task but at sequences of tasks, the metrics have to change too.

The simplest formulation is reward. You define a per-instance reward, reward serves as a proxy for learning, higher is better, and you sum it over the sequence. A system that’s better at continual learning would have higher cumulative reward. That’s true, to be clear. I’m not setting up a fake argument. But the problem with total reward alone is that it confounds continual learning with base model strength.

Picture two systems. The red one is a super smart GPT-X that can do our tasks well. We give it all the tasks in sequence and watch its performance. The black system is a much weaker model to start with, but as it goes through the benchmark it demonstrates better learning capability than the red one. The gray line is what the black system would have scored if every instance were done independently, and you can see that as a function of experience the black system improves. Looking at cumulative reward alone, we’d have concluded the red system is best and missed the black system’s learning entirely.

To resolve this we introduce a gain metric:

gain = stateful reward − stateless reward

In that graph, it’s the black curve minus the gray one, the system measured against its own stateless baseline.

To compute it, you run a system through the sequence twice. The first time you put no limitations on it. All instances run in sequence and there’s shared state the system can maintain, whether that’s context, externalized context, or parametric updates. It can do whatever it wants. The second time you compute the stateless baseline: between every two instances you force a reset, wiping everything and returning the system to its base state.

That lets you control for instance-specific difficulty and initial model capability by measuring whether accumulated experience actually helped. My gain@4 is intuitively how much the prior three instances improved the fourth compared to doing it cold.

You might say, Parth, I could have a weak system that looks great on this because it starts out dumb and improves. That’s valid, which is why I don’t think you can reduce learning to a single metric. We evaluate gain, reward, and cost on Pareto frontiers, because they capture different aspects of what you care about.

Inside a task: database exploration

Let me walk through one task, because it’s a fun one. Database exploration is modeled loosely on what a data engineer does at a company.

You have a bunch of data tables in a large database with various relationships between them. Tables link on particular keys, and there are idiosyncrasies in the data. If you’re a new data engineer on your first day, it’s a massive amount of data and you don’t really know what you’re doing. Any time someone asks you a question, you spend time working it out. As you get more experienced, that changes.

So models are exposed to a vast amount of data, asked questions in natural language, and have to give a final answer by exploring the databases with SQL queries. The first time you ask a cold-started system question one, it runs exploratory queries: what’s the format, how do the tables link, what do I need to answer this? By the tenth time, just like a human, it understands the structure and has figured out some of the idiosyncrasies. Its efficiency, measured in exploratory SQL queries, has gone down quite a bit. You see this even from basic in-context learning systems.

But continual learning isn’t that simple, and companies evolve. You’ve seen database migrations where columns are dropped, renamed, added, and things become defunct. So we also measure concept drift, because you don’t only want to know whether the model is getting better. That’s its plasticity. You also care about stability: retaining old information while adapting to new on the fly.

About halfway through, we migrate the database and ask a completely different set of questions. The model is told there’s a migration, we don’t just drop it out of the loop. But if it indexes purely on past information it gets the answer wrong every time, and it gets feedback that it’s wrong. The question is whether it can reconcile which parts of the old information are still useful against which are stale priors.

We run about 40 tasks in this sequence, then plot learning curves for different systems: gain over time, reward over time, and how both compare as a function of cost.

There are five other tasks in the 1.0 release. Blind spectrum monitoring, related to signal processing. Codebase adaptation, perhaps the most relatable to a technical audience: can a model get more efficient at working in a codebase, so that every time you spin up Claude Code it isn’t doing 50 ls and cat calls to understand your codebase and wasting your tokens? Cohort studies, inspired by epidemiology. Exploitable poker, where you play opponents with exploitable policies, like amateur humans. That one is honestly just because my friends like playing poker. And sales prediction, a more data-science task: as new information arrives, can you update your modeling and forecast better?

Results: naive in-context learning wins

On the leaderboard sorted by cumulative reward, the system with the highest reward also had the highest gain, and it was just in-context learning. We put everything in the model’s context. We didn’t even use the more expensive memory systems. That was the best system, with a Sonnet-class model. The Opus-class models, interestingly, did worse.

The Pareto frontiers are where it gets interesting, because you have to understand how reward and cost, and gain and cost, break down. Perhaps the most surprising result is that in-context learning was dominant on the frontiers much of the time. Systems that were more expensive, used more memory management and more externalized memory, and did more processing offline, did worse.

That was counterintuitive to me. We don’t evaluate parametric systems here, and candidly I don’t think there are many right now that would demonstrate the level of continual learning you need in realistic tasks. The other limitation is GPUs. But on the non-parametric side it’s really surprising that in-context learning beats these more expensive systems, and it suggests there’s a lot of low-hanging fruit still available there in the short term.

Most of the failure modes fall on one side of the stability-plasticity trade-off at the core of continual learning. Stability is retaining prior experience without regressing. Plasticity is adapting to new information and improving on data that may be out of distribution.

Here’s a stability failure from sales prediction. The model gets data about bed frames in New York, makes predictions for a year, and gets a score. Reading the agent’s reasoning: it overpredicted, so it decides to recalibrate and underpredict. Reasonable, a human might do the same. Then it’s told it underpredicted. One number is too high, another too low, so as humans we’d go for something in the middle. The agent goes back to the first prediction and gets told it’s too high again. That looks like a dumb example, but you see it fairly often in the more complex tasks.

Plasticity is a different story. In cohort studies, the model gets data from a bunch of study groups and has to combine it to improve its predictions. Around study 15, the reasoning contains this quote: “these seem to be cohort definitions from a different study schema that don’t apply here.” There was no different study schema. I don’t know what the model was thinking. It was exposed to information slightly out of distribution from what it had seen, and because of that it couldn’t use that information to improve. The paper has more theoretical and grounded analysis of how we measured both.

Rethinking the training stack, and what’s next

I’m personally quite a believer in parametric methods. There’s a lot of low-hanging fruit in non-parametric methods we’ll extract over a one- or two-year horizon, but parametric methods are what will demonstrate real ability in the long run to get native continual learning.

A few years ago the LLM training stack was pre-training, SFT, then RLHF for preference alignment. Then we added a bunch of things. Continued pre-training. Mid-training. RL, except with five different teachers, then multi-teacher on-policy distillation to combine it into one model. The stack looks a lot more Frankenstein, for lack of a better word.

What people do for continual learning right now starts from the observation that we’ve invested billions of dollars training the model to a particular point, so we need methods that work on top of that. There’s a sunk cost assumption baked in, and an assumption that the weights we’ve arrived at are actually susceptible to continual learning, that the features are plastic.

From first principles you wouldn’t build this entire stack and then try to make the model continually learn afterwards. You’d have one phase of training for continual learning, and everything after that is deployment. There’d be one stage where you teach the model how to learn, a meta-learning stage, and after that the model goes off and learns on its own. That’s the purest sense of what learning looks like. We’re not there today, but there’s a lot of exciting research going into it.

Continual learning is one example of how my group is looking to challenge the way we do training. You should always look for ways to challenge the status quo, but more importantly there’s an opportunity to challenge how we do AI research as a whole, which I’ve been writing about.

Continual Learning Bench is not one person. It’s a collaboration with people from Berkeley, UW-Madison, and Snorkel AI, and a wide variety of contributors who worked really hard on it. We were supported by Snorkel through the Open Benchmark Science program, so if you’re interested in building your own benchmark you should check that out, and by the Laude Institute through its Slingshots program.

What’s next is two or three things. More domain-specific and longer-horizon tasks: humans don’t stop after 20 or 40 tasks, we go into hundreds or thousands of instances and still maintain stability. Would in-context learning get worse there? Would parametric methods look better? You have to manage that against the cost of evaluation, so we chose the medium side to begin with. Then there are OSS models and parametric approaches actually designed for the benchmark. And lastly, task environments outside easily verifiable, deterministic domains. Continual learning for personalization is very hard to evaluate deterministically, and user simulation is notoriously hard in language models.

If there’s one thing you take away, it’s that continual learning doesn’t look like point capabilities. We need to measure it the right way to optimize for the right objectives, and I hope this is a first step toward that.

Q&A

Why does simple in-context learning beat a coding agent that can edit its own memory?

I can offer a few pieces of intuition. I’d have to dig into specific failure modes to give a complete answer, but the intuition is that spending a lot more compute on a task in order to learn can actually regress your performance. You can blow up your reasoning budget or your context with potentially unrelated information, which makes it harder to adapt to new information as it arrives.

Say a coding harness externalizes memory to Markdown files, and then new information arrives that it has to update from. That may be harder, because the model already has externalized memory it might be confident in, and editing that is harder. I don’t want to call it context blowup, because I don’t think that’s exactly what it is, but in some form the model has spent more time processing the information and it’s a lot harder for it to adapt. It’s an open question what better design for those harnesses looks like.

Could topological deep learning or very abstract mathematics improve continual learning?

I’m unfamiliar with those specific terms and I’d have to look at it offline. I can’t say I’ve seen approaches using category theory for continual learning, but I’d be very curious to learn more.

Are you suggesting we train from scratch?

Perhaps controversial, but yes, I am suggesting we start training from scratch.

There’s a lot of baggage in the way we do training today. There are assumptions baked in that these methods scale really well, and they do let us reach capabilities we know how to optimize for and that are in distribution. But to develop an actual learning ability inside the model, I think there need to be meta-learning phases before you even do the kind of pre-training we do today.

Right now we have a bunch of data and we’re teaching the model to memorize and learn what’s in front of it. Memorize is a loose word and I’m using it imprecisely. But the objective with learning is to teach the model how to learn, and it’s not clear to me that knowing all this information beforehand makes you a better learner. It’s like having a super-intelligent three-year-old that knows everything in the world but doesn’t know how to learn anything new. I don’t care if you start from scratch with no knowledge. If you have the ability to learn, that makes you a much more useful and much more powerful system.

So I think you do have to start training from scratch. I’m personally interested in different model architectures that would enable that. We might have to step away from what we have today and look at different ways to do training and data curation optimization. It’s an exciting time to still do research here.

Parth Asawa is a PhD student at UC Berkeley working on continual learning, and project lead of Continual Learning Bench, a benchmark developed in collaboration with Snorkel AI. The project was supported by Snorkel’s Open Benchmark Science program and the Laude Institute’s Slingshots program.

Paper: Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments
Benchmark and leaderboard: continual-learning-bench.com
Code: github.com/pgasawa/continual-learning-bench
Also on Benchtalks: Continual Learning Bench with Parth Asawa
Video: Continual Learning Bench: Do AI Systems Learn?

The Snorkel AI Reading Group meets in San Francisco to explore frontier developments in AI with the researchers behind them. See upcoming sessions and RSVP.