Train-to-Test (T²) Scaling Laws: Why Reasoning Models Should Be Overtrained

At our latest Snorkel AI Reading Group, Nicholas Roberts presented Test-Time Scaling Makes Overtraining Compute-Optimal, work coauthored with Sungjun Cho, Zhiqi Gao, Tzu-Heng Huang, Albert Wu, Gabriel Orlanski, Avi Trost, Kelly Buchanan, Aws Albarghouthi, and Frederic Sala.

Pre-training scaling laws answer a familiar question: given a fixed training budget, how should a team allocate compute between model size and training data? Test-time scaling asks a different question: how much inference compute should a model spend generating and verifying candidate answers? In modern reasoning systems, these decisions are coupled—but they have largely been studied in isolation.

The paper introduces Train-to-Test, or T², scaling laws that optimize model size, training tokens, and repeated inference samples together. Across eight downstream tasks, the authors find that once test-time compute is included, the compute-optimal choice shifts sharply toward smaller models trained on far more tokens than standard Chinchilla scaling would recommend. The finding survives post-training and aligns with the broader industry trend toward heavily overtrained small models.

Transcript

Lightly edited for readability.

Thank you for the introduction and for having me here today. I’m going to tell you about recent work on combining pre-training scaling laws with test-time reasoning.

Here’s the roadmap. First, I’ll give some background on pre-training scaling laws: what they look like, why we use them, and what they tell us. Then I’ll discuss test-time scaling, including the most common techniques and how they increase a model’s reasoning capabilities. Finally, I’ll explain how we combine pre-training and test-time scaling in an end-to-end scaling law. The result is surprising: if a model will use test-time reasoning, it can be optimal to overtrain it for far longer than standard pre-training scaling would suggest.

What are pre-training scaling laws?

Suppose you have a fixed amount of compute and need to train a large language model. Perhaps you have access to a large data center for exactly one week and want to train the best model possible. You have a couple of choices.

You could train a very large model, but larger models require more compute, so you cannot train them for as long. Alternatively, you could train on much more data, but that also costs compute, so you would have to use a smaller model. In practice, you want to do both: increase model size and train on more data. A scaling law gives you a roadmap for allocating a fixed compute budget between those two dimensions.

One of the most famous examples is Chinchilla scaling. The basic approach is to train many smaller models and use the results to estimate a parametric function of loss as a function of model size and dataset size. At smaller scales, you identify the combination of parameters and training tokens that minimizes loss under a fixed compute budget. You repeat that analysis across budgets and then forecast the optimal trade-off at much larger scales.

The rule of thumb associated with Chinchilla is roughly 20 training tokens per parameter. If you cannot run the full scaling analysis yourself, that ratio provides a practical default, and it has become a standard reference point for pre-training.

On a Chinchilla IsoFLOP plot, each curve represents one fixed compute budget. Every point on a curve represents a different trade-off between parameter count and dataset size. At one extreme, a very large model can only be trained briefly. At the other, a small model can be trained much longer, but may lack sufficient capacity. The minimum of each curve identifies the best trade-off for that budget, and those minima follow a power law as compute increases.

That is the standard pre-training picture. But modern models increasingly use reasoning at inference time. This raises a new question: are we pre-training models suboptimally if we know they will use test-time scaling?

Test-time scaling for reasoning

There are many ways to implement reasoning in language models: chain-of-thought prompting, self-consistency, Monte Carlo tree search, reward-model-guided search, iterative refinement, rejection sampling, and parallel sampling.

We focus on parallel sampling because it is simple, often competitive, and comparatively easy to analyze. In parallel sampling, you prompt a model, draw multiple independent completions, verify them, and select a completion that passes the verifier.

Performance is commonly measured with pass@k. If you draw k completions, the probability that at least one is correct should increase with k. For a problem with single-sample success probability p, pass@k is:

$$\operatorname{pass@}k = 1-(1-p)^k.$$

As the number of samples increases, the probability of seeing a correct answer at least once increases. This gives us the test-time component we want to connect to pre-training.

From Chinchilla to Train-to-Test scaling

The Chinchilla scaling law models loss as a function of model size, N, and dataset size, D:

$$L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta}.$$

The first power-law term captures the effect of model capacity, the second captures the effect of training data, and E is the asymptotic loss floor—the best loss the setup can approach as model size and dataset size go to infinity.

We add a third dimension: k, the number of repeated samples used at test time. Our first approach extends the loss model with another power-law term:

$$\widehat{L}(N,D,k)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta}+\frac{G}{k^\gamma}.$$

We also correct for inference cost. A forward pass through a larger model costs more than a forward pass through a smaller one. To compare models fairly, we hold the inference budget constant and allocate more samples to smaller models. Using the approximation

$$C_{\mathrm{inf}}=2Nk,$$

the number of samples is

$$k=\frac{C_{\mathrm{inf}}}{2N}.$$

This lets a small model use more repeated samples while keeping its total inference cost equal to that of a larger model.

There is a subtlety here. Chinchilla models negative log-likelihood, while repeated sampling is usually evaluated with pass@k, an accuracy-like metric. Why is it valid to put k into a loss model?

The standard negative log-likelihood objective is the expected negative log probability of the correct answer. Under repeated sampling, we can instead model the negative log probability of obtaining a correct answer at least once:

$$\mathbb{E}\left[-\log\left(1-(1-p)^k\right)\right].$$

That gives us a loss-style objective that incorporates repeated sampling.

We also developed a second approach that models pass@k directly. It uses a Beta regression to represent the distribution of per-question success probabilities and then derives expected pass@k from that distribution. The key point is that we have two distinct models: one parameterizes loss and the other parameterizes accuracy. Although they model different quantities, both produce the same qualitative conclusion.

Reasoning changes the compute-optimal training regime

Under standard Chinchilla scaling, optimal tokens per parameter remain near the familiar rule of thumb. Under T² scaling, the forecast changes radically. Once we account for test-time reasoning and equalize inference cost, optimal tokens per parameter grow dramatically. The compute-optimal choice moves toward smaller models that are trained on substantially more data.

At first, that result looks concerning because it implies a much greater demand for high-quality training data. We already know that clean internet data is limited, and training frontier models is expensive.

But on the day we finished the paper, Liquid AI released LFM2.5-350M, a 350-million-parameter model trained on 28 trillion tokens. That is about 80,000 tokens per parameter—far beyond the Chinchilla rule of thumb of roughly 20. People reacted by saying that standard Chinchilla scaling was dead. Our T² forecast, however, places a model like this close to the region it predicts for inference-efficient reasoning systems.

When we plot the Liquid model alongside our forecasts, it falls near the T² trend. Our loss-based model and our accuracy-based model do not produce identical curves, but both point in the same direction: substantial overtraining relative to the standard pre-training allocation.

Validating the forecast

We wanted to know whether the scaling law could predict genuinely overtrained checkpoints, rather than merely fit the data used to construct it. We therefore held out heavily overtrained models, fit the scaling law without them, and tested whether the model could extrapolate into that regime.

The held-out checkpoints fell broadly on trend. The loss-based approach extrapolated somewhat better than the accuracy-based approach, so practitioners may prefer the parametric loss model in this setting.

We also compared the best model under joint training and test-time scaling with a Chinchilla-optimal model given the same inference budget. Across real and synthetic benchmarks, the best overtrained checkpoint consistently and substantially outperformed the Chinchilla-optimal checkpoint.

The IsoFLOP curves make the change visible. Once test-time scaling is included, the curves warp and shift downward, and the optimal frontier moves toward smaller, more heavily trained models. Again, the loss-based and accuracy-based approaches differ quantitatively but agree qualitatively.

Does the finding survive post-training?

In practice, nobody deploys test-time scaling only on base models. Models usually undergo post-training—such as supervised fine-tuning or reinforcement learning—before reasoning is applied at deployment.

We tested whether the T² recommendation survives two stylized forms of post-training: ordinary fine-tuning and supervised fine-tuning with prompt tokens masked from the loss. The finding persisted. After post-training, T² still recommended significant overtraining compared with Chinchilla, and the best overtrained checkpoints continued to outperform Chinchilla-optimal checkpoints under the same inference budget.

Industry has qualitatively known for some time that small reasoning models benefit from heavy overtraining. Recent small-model releases are evidence of that. But the practice has largely been ad hoc. This work provides a scientific scaling law for making the pre-training and test-time allocation jointly.

That brings me to the end. Thank you.

Q&A

How does T² account for differences in pass@k behavior across models?

Question: Pass@k may scale differently with k for different models. It depends on how concentrated the output distribution is, which in turn depends on architecture and training. How does the model account for that?

Nicholas Roberts: We assume a stylized setting with a single correct answer for each evaluation example. If an evaluation contains very difficult problems with many valid solutions, the per-sample probability p can be difficult to estimate. We therefore evaluate on simpler datasets with a clearly defined correct answer.

Differences between models are still represented in the scaling law. For example, a smaller model may begin with a lower pass@1 than a larger model. We scale the number of samples for each model to match inference cost, and the model predicts how performance changes across those configurations.

Follow-up: A more heavily trained or instruction-tuned model may have better performance at small k but a more concentrated output distribution, potentially giving it worse pass@k at large k. That suggests a trade-off between post-training compute and inference-time compute.

Nicholas Roberts: I see what you mean. In our post-training experiments, we used the same post-training methodology for every checkpoint. One could extend this into a train-to-post-train-to-test scaling law that explicitly includes reinforcement learning or supervised fine-tuning. Post-training should increase the probability of correct answers on evaluations aligned with the post-training method, but we do not model that allocation explicitly here. That may be the next paper.

Does the Beta regression control for heterogeneous task difficulty?

Question: Beta regression generally assumes IID samples. In scaling experiments, evaluation subsets may differ in difficulty. Do you control for that?

Nicholas Roberts: We chose Beta regression because prior pass@k literature compared several distributions for modeling per-question probabilities and found that the Beta distribution gave a strong empirical fit. Problem difficulty is heterogeneous, and that heterogeneity is one reason to use the Beta model rather than simply applying a sigmoid to the mean loss. The fitted distribution captures variation in difficulty across problems.

Did you retrain checkpoints or use an existing scaling suite?

Question: How do you decide whether a checkpoint needs retraining rather than a data-quality or process fix?

Nicholas Roberts: We did not retrain checkpoints in that sense. We began with an off-the-shelf, Chinchilla-style scaling suite trained on a standard pre-training distribution, then extended it with additional overtrained checkpoints.

Could differences in training data explain the improvement?

Question: Smaller models are trained on more tokens than larger models. When you compare an overtrained checkpoint with a Chinchilla-optimal checkpoint, the models may have seen different token samples. Could the results be explained by the data rather than the proposed allocation?

Nicholas Roberts: At a fixed compute budget, larger models are necessarily trained for fewer tokens because each forward and backward pass costs more. Smaller models can be trained for longer. That is already true in standard Chinchilla scaling. Our result is that the optimal trade-off shifts much further toward small-model overtraining once test-time compute is included.

The scaling curves remain smooth within each IsoFLOP budget. If different token samples were introducing a major confound, we would expect unpredictable high or low points rather than smooth scaling. Like other pre-training scaling laws, the analysis assumes a consistent hypothesis class—meaning the same architecture family and hyperparameter methodology—and the same data distribution. If the data distribution changed, it could move a checkpoint away from the predicted trend. We do not observe that here.

Why not optimize marginal compute benefits directly?

Question: You are minimizing loss with respect to N, D, and k. Why not equalize the marginal benefit per unit of compute for each variable, account for the constraints with a Lagrangian, and estimate the derivatives empirically?

Nicholas Roberts: You could do that, although I think you would still need a parametric model.

Follow-up: You could assume smoothness, which the current method already does.

Nicholas Roberts: We did not explore that approach, but it is an interesting direction.

Previous Reading Groups

• Senior SWE-Bench: Evaluating Coding Agents Like Senior Engineers
• Agents’ Last Exam: AI Benchmarking for Real Work
• JudgmentBench: Comparing Rubric and Preference Evaluation for Quality Assessment
• Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaboration
• Code World Models and AutoHarness for LLM Agents
• Olmix: Data Mixing for LM Development

Nicholas Roberts recently completed his PhD at the University of Wisconsin–Madison and joins Princeton University this fall as a postdoctoral research fellow. He has previously worked at Jane Street, Meta’s Llama team, Together AI, and Microsoft Research.

Read the paper: Test-Time Scaling Makes Overtraining Compute-Optimal. The work will be featured at COLM 2026 and was covered by VentureBeat: Train-to-test scaling explained.

The Snorkel AI Reading Group meets regularly in San Francisco to discuss frontier AI research. See upcoming sessions.