Grok 4.5 Testing Results: How SpaceXAI’s New Model Performs on Real Professional Work

We’ve evaluated Grok 4.5 on Snorkel’s GDPval+ dataset, Snorkel’s expert-created dataset of professional workplace reasoning tasks from across the economy. To compare performance against other frontier models, we ran the evaluation alongside GPT 5.5 and Claude Opus 4.8. Overall, Grok 4.5 demonstrated the strongest overall performance.

Dataset

GDPval+ is part of the Snorkel Data Series (SDS), Snorkel’s portfolio of expert-curated datasets. GDPval+ is for training and evaluating AI models on professional work tasks across the U.S. economy. Tasks are authored by domain experts who create the prompts and produce real workplace deliverables (documents, spreadsheets, presentations). The following analysis is from a ~2,000-task sample of the total GDPval+ Edition 1 dataset. Task occupation and sector distributions are illustrated in the figures below.

Methodology

We run our benchmarking via Harbor, the open-source framework for LLM/agent evaluations. Models attempt each task once (k = 1), writing their final output documents to a given directory. Verifiers are automatically triggered upon task completion, utilizing LLMaJ (with Gemini 3.5 Flash) and each task’s expert-created rubric to grade the model outputs. A model achieves a “pass” when it obtains a perfect reward score (r = 1.0) on the task’s rubric.

Each model is coupled with an agent harness that can call a variety of tools, create/process documents, and search the web. In this analysis, GPT 5.5 & Opus 4.8 are coupled with a Stirrup evaluation agent tuned for performance on GDPVal-style tasks (which we have observed achieves higher performance than each model’s respective proprietary Harbor agent). At SpaceXAI’s request Grok 4.5 is coupled with the Grok Build agent (customized such that it can run within Harbor).

Results: Mean pass rates

Grok 4.5 achieves a mean pass rate of 29% on the ~2,000 Snorkel GDPVal+ tasks used in this evaluation (figure above), surpassing the mean pass rates achieved by GPT 5.5 (22%) and Opus 4.8 (21%). The improvement is concentrated in domains that demand deep professional judgment (figures below): legal work (40% vs 27–28%), education (58% vs 35–42%), healthcare (35% vs 23–25%), and QA analysis (37% vs 19–27%). GPT 5.5 led on construction and Opus 4.8 led among financial managers — but across most occupations and sectors, Grok 4.5 led, often by wide margins.

Results: Failure mode analysis

To perform our failure analysis, we utilize an LLM to derive a failure taxonomy from every failed criterion (for each trial) – we then assign each failed criterion a tag (derived from the taxonomy) denoting its failure type. Grok 4.5 showed the lowest prevalence in all six error categories we track (figure above). Missing domain analysis — the most common failure for every model — dropped to 40% of samples versus 51–52% with GPT and Opus. Grok 4.5 also achieved a notably lower percentage of incorrect recommendation, format & structure, and missing source references errors.

Grok 4.5’s lower error rates are clearly demonstrated in its output deliverables. On a wage-and-hour liability task, Grok 4.5 was the only model to satisfy every expert criterion, correctly handling a legal nuance in the rate calculation that the others missed. On a commercial real estate task, it produced differentiated, site-level recommendations where others returned generic, uniform guidance.

Conclusions

Expert-level work remains an open frontier — even the best models pass fewer than one in three expert criteria — which makes Grok 4.5’s performance increases over other frontier models that much more meaningful. On this set of ~2,000 of Snorkel’s GDPVal+ tasks, Grok 4.5 achieves stronger mean pass rates, outperforms other models across many sectors/occupations, and obtains lower error rates.

At Snorkel, we are excited not only to be a provider of expert-quality data, but to also be a trusted partner in shaping and evaluating the frontier of Artificial Intelligence. We will continue to update the community and share detailed performance findings on our datasets as newer frontier models are released.

Learn more about GDPval+.