New Snorkel benchmark leaderboards. See the results.
All Topics
Evaluating AI Agents for Insurance Underwriting
In this post, we will show you a specialized benchmark dataset we developed with our expert network of Chartered Property and Casualty Underwriters (CPCUs). The benchmark uncovers several model-specific and actionable error modes, including basic tool use errors and a surprising number of insidious hallucinations from one provider. This is part of an ongoing series of benchmarks we are releasing across verticals…
Our picks
Building the Data Development Platform for Specialized AI
Announcing two new products on our AI Data Development Platform that together create a complete solution for enterprises to specialize AI systems with expert data at scale.
Why enterprise GenAI evaluation requires fine-grained metrics to be insightful
GenAI needs fine-grained evaluation for AI teams to gain actionable insights.
Download and read "The Guide to Data Labeling & Annotation for Enterprise AI Teams"
Latest posts
Building the Benchmark: Inside Our Agentic Insurance Underwriting Dataset
In this post, we unpack how Snorkel built a realistic benchmark dataset to evaluate AI agents in commercial insurance underwriting. From expert-driven data design to multi-tool reasoning tasks, see how our approach surfaces actionable failure modes that generic benchmarks miss—revealing what it really takes to deploy AI in enterprise workflows.
Evaluating AI Agents for Insurance Underwriting
In this post, we will show you a specialized benchmark dataset we developed with our expert network of Chartered Property and Casualty Underwriters (CPCUs). The benchmark uncovers several model-specific and actionable error modes, including basic tool use errors and a surprising number of insidious hallucinations from one provider. This is part of an ongoing series of benchmarks we are releasing across verticals…
LLM Observability: Key Practices, Tools, and Challenges
LLM observability is crucial for monitoring, debugging, and improving large language models. Learn key practices, tools, and strategies of LLM observability.
Anthropic Claude + AWS: revolutionizing pharma data analytics with Snorkel AI
Explore how Anthropic Claude + AWS help pharmaceutical companies leverage AI for enhanced data insights and revenue growth.
Latest videos
Expert Data, Specialized AI: Snorkel Summer 2025 Launch Event
Why Labeled Data is Essential for Generative AI Success
Enhancing LLM Observability and Gen AI Evaluation for Business Success
How to Evaluate LLM Performance for Domain-Specific Use Cases
Understand the basics of LLM training in under four minutes!
RAG Optimization: A Practical Overview for Improving Retrieval Augmented Generation
Applied AI
Why GenAI evaluation requires SME-in-the-loop for validation and trust
It’s critical enterprises can trust and rely on GenAI evaluation results, and for that, SME-in-the-loop workflows are needed. In my first blog post on enterprise GenAI evaluation, I discussed the importance of specialized evaluators as a scalable proxy for SMEs. It simply isn’t practical to task SMEs with performing manual evaluations – it can take weeks if not longer, unnecessarily…
Research spotlight: is long chain-of-thought structure all that matters when it comes to LLM reasoning distillation?
We’re taking a look at the research paper, LLMs can easily learn to reason from demonstration (Li et al., 2025), in this week’s community research spotlight. It focuses on how the structure of reasoning traces impacts distillation from models such as DeepSeek R1. What’s the big idea regarding LLM reasoning distillation? The reasoning capabilities of powerful models such as DeepSeek…
Why enterprise GenAI evaluation requires fine-grained metrics to be insightful
GenAI needs fine-grained evaluation for AI teams to gain actionable insights.
What is specialized GenAI evaluation, and why is it so critical to enterprise AI?
Specialized GenAI evaluation ensures AI assistants meet business requirements, SME expertise, and industry regulations—critical for production-ready AI.
Customers
Anthropic Claude + AWS: revolutionizing pharma data analytics with Snorkel AI
Explore how Anthropic Claude + AWS help pharmaceutical companies leverage AI for enhanced data insights and revenue growth.
Call center AI for customer experience management: a case study
How one large financial institution used call center AI to inform customer experience management with real-time data.
How we achieved 89% accuracy on contract question answering
A customer wanted an llm system for complex contract question answering tasks. We helped them build it—beating the baseline by 64 points.
Content filtering breakthrough: Snorkel client reaches 96% recall in 3 days
Snorkel AI helped a client solve the challenge of social media content filtering quickly and sustainably. Here’s how.
Data development
Building the Benchmark: Inside Our Agentic Insurance Underwriting Dataset
In this post, we unpack how Snorkel built a realistic benchmark dataset to evaluate AI agents in commercial insurance underwriting. From expert-driven data design to multi-tool reasoning tasks, see how our approach surfaces actionable failure modes that generic benchmarks miss—revealing what it really takes to deploy AI in enterprise workflows.
Evaluating AI Agents for Insurance Underwriting
In this post, we will show you a specialized benchmark dataset we developed with our expert network of Chartered Property and Casualty Underwriters (CPCUs). The benchmark uncovers several model-specific and actionable error modes, including basic tool use errors and a surprising number of insidious hallucinations from one provider. This is part of an ongoing series of benchmarks we are releasing across verticals…
LLM Observability: Key Practices, Tools, and Challenges
LLM observability is crucial for monitoring, debugging, and improving large language models. Learn key practices, tools, and strategies of LLM observability.
LLM-as-a-judge for enterprises: evaluate model alignment at scale
Discover how enterprises can leverage LLM-as-Judge systems to evaluate generative AI outputs at scale, improve model alignment, reduce costs, and tackle challenges like bias and interpretability.
Product
Data-Centric Development of an Enterprise AI Agent with Snorkel
See how we can use these two new products—Snorkel Evaluate and Expert Data-as-a-Service–to evaluate and develop a specialized agentic AI system for an enterprise use case
Building the Data Development Platform for Specialized AI
Announcing two new products on our AI Data Development Platform that together create a complete solution for enterprises to specialize AI systems with expert data at scale.
Databricks + Snorkel Flow: integrated, streamlined AI development
Discover the power of integrating Databricks and Snorkel Flow for efficient data ingestion, labeling, model development, and AI deployment.
How LLM evaluation drives better models in Snorkel Flow
Discover how Snorkel AI’s methodical workflow can simplify the evaluation of LLM systems. Achieve better model performance in less time.
Research
Research spotlight: is long chain-of-thought structure all that matters when it comes to LLM reasoning distillation?
We’re taking a look at the research paper, LLMs can easily learn to reason from demonstration (Li et al., 2025), in this week’s community research spotlight. It focuses on how the structure of reasoning traces impacts distillation from models such as DeepSeek R1. What’s the big idea regarding LLM reasoning distillation? The reasoning capabilities of powerful models such as DeepSeek…
Research spotlight: Is intent analysis the key to unlocking more accurate LLM question answering?
Learn how ARR improves QA accuracy in LLMs through intent analysis, retrieval, and reasoning. Is intent the key to smarter AI? Explore ARR results!
Long context models in the enterprise: benchmarks and beyond
Snorkel researchers devised a new way to evaluate long context models and address their “lost-in-the-middle” challenges with mediod voting.
How ROBOSHOT boosts zero-shot foundation model performance
ROBOSHOT acts like a lens on foundation models and improves their zero-shot performance without additional fine-tuning.
Trending stories
1
Foundation Models (FMs), such as GPT-3 and Stable Diffusion, mark the beginning of a new era in machine learning and artificial intelligence. What are they and how will they impact your business? Find out in our guide.
2
Announcing two new products on our AI Data Development Platform that together create a complete solution for enterprises to specialize AI systems with expert data at scale.
3
We started the Snorkel project at the Stanford AI lab in 2015 around two core hypotheses:
4
Machine Learning Whiteboard (MLW) Open-source Series Today, Ryan Smith, machine learning research engineer at Snorkel AI, talks about prompting methods with language models and some applications they have with weak supervision. In this talk, we’re essentially going to be using this paper as a template—this paper is a great survey over some methods in prompting from the last few years…
5
The Future of Data-Centric AI Talk Series Background Roshni Malani received her PhD in Software Engineering from the University of California, San Diego, and has previously worked on Siri at Apple and as a founding engineer for Google Photos. She gave a presentation at the Future of Data-Centric AI virtual conference in September 2021. Her presentation is below, lightly edited…
6
We’re excited to announce the Q4 2021 LTS release of Snorkel Flow, our data-centric AI development platform powered by programmatic labeling. This latest release introduces a number of new product capabilities and enhancements, from a streamlined programmatic data development interface, to enhanced auto-suggest for labeling functions, to new machine learning capabilities like AutoML, to significant performance enhancements for PDF data…
Latest posts
Building the Benchmark: Inside Our Agentic Insurance Underwriting Dataset
In this post, we unpack how Snorkel built a realistic benchmark dataset to evaluate AI agents in commercial insurance underwriting. From expert-driven data design to multi-tool reasoning tasks, see how our approach surfaces actionable failure modes that generic benchmarks miss—revealing what it really takes to deploy AI in enterprise workflows.
Evaluating AI Agents for Insurance Underwriting
In this post, we will show you a specialized benchmark dataset we developed with our expert network of Chartered Property and Casualty Underwriters (CPCUs). The benchmark uncovers several model-specific and actionable error modes, including basic tool use errors and a surprising number of insidious hallucinations from one provider. This is part of an ongoing series of benchmarks we are releasing across verticals…
LLM Observability: Key Practices, Tools, and Challenges
LLM observability is crucial for monitoring, debugging, and improving large language models. Learn key practices, tools, and strategies of LLM observability.
Anthropic Claude + AWS: revolutionizing pharma data analytics with Snorkel AI
Explore how Anthropic Claude + AWS help pharmaceutical companies leverage AI for enhanced data insights and revenue growth.
Data-Centric Development of an Enterprise AI Agent with Snorkel
See how we can use these two new products—Snorkel Evaluate and Expert Data-as-a-Service–to evaluate and develop a specialized agentic AI system for an enterprise use case
Building the Data Development Platform for Specialized AI
Announcing two new products on our AI Data Development Platform that together create a complete solution for enterprises to specialize AI systems with expert data at scale.
LLM-as-a-judge for enterprises: evaluate model alignment at scale
Discover how enterprises can leverage LLM-as-Judge systems to evaluate generative AI outputs at scale, improve model alignment, reduce costs, and tackle challenges like bias and interpretability.
Why GenAI evaluation requires SME-in-the-loop for validation and trust
It’s critical enterprises can trust and rely on GenAI evaluation results, and for that, SME-in-the-loop workflows are needed. In my first blog post on enterprise GenAI evaluation, I discussed the importance of specialized evaluators as a scalable proxy for SMEs. It simply isn’t practical to task SMEs with performing manual evaluations – it can take weeks if not longer, unnecessarily…
Research spotlight: is long chain-of-thought structure all that matters when it comes to LLM reasoning distillation?
We’re taking a look at the research paper, LLMs can easily learn to reason from demonstration (Li et al., 2025), in this week’s community research spotlight. It focuses on how the structure of reasoning traces impacts distillation from models such as DeepSeek R1. What’s the big idea regarding LLM reasoning distillation? The reasoning capabilities of powerful models such as DeepSeek…
Why enterprise GenAI evaluation requires fine-grained metrics to be insightful
GenAI needs fine-grained evaluation for AI teams to gain actionable insights.
What is specialized GenAI evaluation, and why is it so critical to enterprise AI?
Specialized GenAI evaluation ensures AI assistants meet business requirements, SME expertise, and industry regulations—critical for production-ready AI.
LLM alignment techniques: 4 post-training approaches
Ensure your LLMs align with your values and goals using LLM alignment techniques. Learn how to mitigate risks and optimize performance.
Research spotlight: Is intent analysis the key to unlocking more accurate LLM question answering?
Learn how ARR improves QA accuracy in LLMs through intent analysis, retrieval, and reasoning. Is intent the key to smarter AI? Explore ARR results!
Why enterprises should embrace LLM distillation
Unlock possibilities for your enterprise with LLM distillation. Learn how distilled, task-specific models boost performance and shrink costs.
Retrieval-augmented generation (RAG) failure modes and how to fix them
Discover common RAG failure modes and how to fix them. Learn how to optimize retrieval-augmented generation systems for max business value.
What is large language model (LLM) alignment?
Learn about large language model (LLM) alignment and how it maximizes the effectiveness of AI outputs for organizations.
Databricks + Snorkel Flow: integrated, streamlined AI development
Discover the power of integrating Databricks and Snorkel Flow for efficient data ingestion, labeling, model development, and AI deployment.
How LLM evaluation drives better models in Snorkel Flow
Discover how Snorkel AI’s methodical workflow can simplify the evaluation of LLM systems. Achieve better model performance in less time.
Unlock proprietary data with Snorkel Flow and Amazon SageMaker
Accelerate LLM development with Snorkel Flow and SageMaker. Automate dataset curation, accelerate training, and gain a competitive advantage.
LLM evaluation in enterprise applications: a new era in ML
Learn about the obstacles faced by data scientists in LLM evaluation and discover effective strategies for overcoming them.
Snorkel AI joins the AWS ISV Accelerate Program and launches Snorkel Flow Availability in AWS Marketplace
Snorkel AI and AWS are partnering to help enterprises build, deploy, and evaluate custom, production-ready AI models. Learn how.
AI data development: a guide for data science projects
What is AI data development? AI data development includes any action taken to convert raw information into a format useful to AI.
SnorkelCon 2024: Inaugural Snorkel AI user conference gathers leaders from 30+ Fortune 500 companies
Discover highlights of Snorkel AI’s first annual SnorkelCon user conference. Explore Snorkel’s programmatic AI data development achievements.
Explore the new GenAI Evaluation Suite: Snorkel 2024.R3
We aim to help our customers get GenAI into production. In our 2024.R3 release, we’ve delivered some exciting GenAI evaluation results.
Snorkel Flow 2024.R3: Supercharge your AI development with enhanced data-centric workflows
Snorkel AI has made building production-ready, high-value enterprise AI applications faster and easier than ever. The 2024.R3 update to our Snorkel Flow AI data development platform streamlines data-centric workflows, from easier-than-ever generative AI evaluation to multi-schema annotation.
New NLP features in Snorkel Flow 2024.R3
Discover new NLP features in Snorkel Flow\’s 2024.R3 release, including named entity recognition for PDFs + advanced sequence tagging tools.
Enterprise data compliance and security review: Snorkel Flow 2024.R3
Discover the latest enterprise readiness features for Snorkel Flow. Configure safeguards for data compliance and security.
How a global financial services company built a specialized AI copilot accurate enough for production
Learn how Snorkel, Databricks, and AWS enabled the team to build and deploy small, specialized, and highly accurate models which met their AI production requirements and strategic goals.
Task Me Anything: innovating multimodal model benchmarks
“Task Me Anything” empowers data scientists to generate bespoke benchmarks to assess and choose the right multimodal model for their needs.
Alfred: Data labeling with foundation models and weak supervision
Introducing Alfred: an open-source tool for combining foundation models with weak supervision for faster development of academic data sets.
Join our newsletter for expert advice, the latest research, and exclusive events.
By submitting this form, I acknowledge I will receive email updates from Snorkel AI, and I agree to the Terms of Use and acknowledge that my information will be used in accordance with the Privacy Policy.