We develop methods, benchmarks, and training systems that turn expert data into frontier AI
building benchmarks and collaborating with
Featured research
Senior SWE-bench built by Snorkel with Princeton University and UW-Madison
Continual Learning Bench: Evaluating agents that adapt and improve over time
Learning from Less: Measuring the Effectiveness of RLVR in Low Data Compute Regimes
Agents' Last Exam: Challenge and measure AI agents on economically valuable and real-world tasks
SlopCodeBench: A community benchmark measuring code erosion
Harvey’s BigLaw Bench: Research
Terminal-Bench 2.0: Raising the bar for AI agent evaluation
Benchmarking Agents in Insurance Underwriting Environments
Agentic Coding benchmark: Evaluating AI Models on complex, real-world coding tasks
Vision and impact
We help labs advance frontier models by working with domain experts to design and build complex, realistic datasets that drive model performance.
Benchmarking & Evaluation
Build benchmarks that define and advance the AI frontier
Scaling Subject Matter Expertise
Define how subject matter experts encode their knowledge into data
RL, Training, & Data Valuation
Drive dataset development based on feedback from RL and model training
Community and open science
Open benchmarks, conversations, and research for real-world AI performance.
Open Benchmarks Grants
Backed by a $3M commitment, the program funds open-source datasets, benchmarks, and evaluation artifacts that shape how frontier AI systems are built and evaluated.
Benchtalks
Reading Group
DEEP RESEARCH Expertise
Technical advisors and distinguished affiliates
Browse research blogs and academic papers
Gideon Mann, head of ML Product and Research at Bloomberg LP, chatted with Snorkel CEO Alex Ratner about building BloombergGPT.
Professionals in the data science space often debate whether RAG or fine-tuning yields the better result. The answer is “both.”
We conducted research to reduce the amount of labeled data required to train machine learning systems. The pinnacle of this effort is the development of TAGLETS, a machine learning system that seamlessly integrates widely known collections of labeled data with a diverse array of machine learning algorithms, known as weak labelers. The system’s evolution has been significantly influenced by comprehensive…
Past U.S. Chief Data Scientist DJ Patil talked with Snorkel AI CEO Alex Ratner on topics including the origin of the title “data scientist.”
The surest way to improve foundation models is through more and better data, but Snorkel researchers showed FMs can learn from themselves.
Generative AI can write poems, recite common knowledge, and extract information. GenAI can also help quickly build predictive pipelines.
Users and organizations are generating ever-increasing amounts of private data from a wide range of sources. Incorporating private context is important to personalize open-domain tasks such as question-answering, fact-checking, and personal assistants. State-of-the-art systems for these tasks explicitly retrieve information that is relevant to an input question from a background corpus before producing an answer. While today’s retrieval systems assume…
Getting better performance from foundation models (with less data)
GenAI may be the most transformative technology of the past decade but data is where enterprises are able to realize real value from AI today.
