In this post, we unpack how Snorkel built a realistic benchmark dataset to evaluate AI agents in commercial insurance underwriting. From expert-driven data design to multi-tool reasoning tasks, see how our approach surfaces actionable failure modes that generic benchmarks miss—revealing what it really takes to deploy AI in enterprise workflows.

In this post, we will show you a specialized benchmark dataset we developed with our expert network of Chartered Property and Casualty Underwriters (CPCUs). The benchmark uncovers several model-specific and actionable error modes, including basic tool use errors and a surprising number of insidious hallucinations from one provider. This is part of an ongoing series of benchmarks we are releasing across verticals…

Discover how enterprises can leverage LLM-as-Judge systems to evaluate generative AI outputs at scale, improve model alignment, reduce costs, and tackle challenges like bias and interpretability.

Unlock possibilities for your enterprise with LLM distillation. Learn how distilled, task-specific models boost performance and shrink costs.

Learn about the obstacles faced by data scientists in LLM evaluation and discover effective strategies for overcoming them.

What is AI data development? AI data development includes any action taken to convert raw information into a format useful to AI.

Learn how Snorkel, Databricks, and AWS enabled the team to build and deploy small, specialized, and highly accurate models which met their AI production requirements and strategic goals.

“Task Me Anything” empowers data scientists to generate bespoke benchmarks to assess and choose the right multimodal model for their needs.

Introducing Alfred: an open-source tool for combining foundation models with weak supervision for faster development of academic data sets.

This release features new GenAI tools and Multi-Schema Annotation, as well as new enterprise security tools and an updated home page.

Enterprises must evaluate LLM performance for production deployment. Custom, automated eval + data slices present the best path to production.

Meta’s Llama 3.1 405B, rivals GPT-4o in benchmarks, offering powerful AI capabilities. Despite high costs, it can enhance LLM adoption through fine-tuning, distillation, and as an AI judge.

Meta released Llama 3 405B today, signaling a new era of open source AI. The model is ready to use on Snorkel Flow.

High-performing AI systems require more than a well-designed model. They also require properly constructed training and testing data.