We develop methods, benchmarks, and training systems that turn expert data into frontier AI
building benchmarks and collaborating with
Featured research
Senior SWE-bench built by Snorkel with Princeton University and UW-Madison
Continual Learning Bench: Evaluating agents that adapt and improve over time
Learning from Less: Measuring the Effectiveness of RLVR in Low Data Compute Regimes
Agents' Last Exam: Challenge and measure AI agents on economically valuable and real-world tasks
SlopCodeBench: A community benchmark measuring code erosion
Harvey’s BigLaw Bench: Research
Terminal-Bench 2.0: Raising the bar for AI agent evaluation
Benchmarking Agents in Insurance Underwriting Environments
Agentic Coding benchmark: Evaluating AI Models on complex, real-world coding tasks
Vision and impact
We help labs advance frontier models by working with domain experts to design and build complex, realistic datasets that drive model performance.
Benchmarking & Evaluation
Build benchmarks that define and advance the AI frontier
Scaling Subject Matter Expertise
Define how subject matter experts encode their knowledge into data
RL, Training, & Data Valuation
Drive dataset development based on feedback from RL and model training
Community and open science
Open benchmarks, conversations, and research for real-world AI performance.
Open Benchmarks Grants
Backed by a $3M commitment, the program funds open-source datasets, benchmarks, and evaluation artifacts that shape how frontier AI systems are built and evaluated.
Benchtalks
Reading Group
DEEP RESEARCH Expertise
Technical advisors and distinguished affiliates
Browse research blogs and academic papers
Brown professor Stephen Bach tells Snorkel CEO Alex Ratner about his research into improving foundation models like GPT-3 with curated data.
Cleanlab Co-Founder and CEO Curtis Northcutt presents his company’s automatic, universal and open-source tools to quickly clean data sets.
Anirudh Koul is Machine Learning Lead for the NASA Frontier Development Lab and the Head of Machine Learning Sciences at Pinterest. He presented at Snorkel AI’s 2022 Future of Data Centric AI (FDCAI) Conference.
Most poll respondents at Snorkel AI’s recent Foundation Model Virtual Summit named questionable accuracy as the biggest barrier preventing them from getting organizational value from Foundation Models.
Snorkel CEO Alex Ratner interviews Mayee Chen about how Liger improves the effectiveness of programmatic labeling through foundation model embeddings.
Hamsa Bastani presented a summary of her and her co-authors’ ongoing work using machine learning and Snorkel AI’s tools to detect and track activities that are associated with a high risk for global sex trafficking.
Snorkel AI co-founder and CEO Alex Ratner recently interviewed several Snorkel researchers about their published academic papers. In this video, Alex talks with Ryan Smith, Senior Applied Scientist at Snorkel, about the work he did on using foundation models to build compact, deployable, and effective models.
Snorkel AI held its Foundation Model Summit Jan 17, bringing together 12 presenters and over 600 attendees at 10 virtual sessions. The event drew registrants from across many sectors, including the tech industry, healthcare, and financial services.
Snorkel AI co-founder and CEO Alex Ratner talks with Ananya Kumar about the work he did on improving the effectiveness of foundation models by using contrastive learning, image augmentations, and labeled subsamples.
A one-day, invite-only summit providing a first look at the benchmarks and research that will shape the frontier.
