LLMs

In this post, we will show you a specialized benchmark dataset we developed with our expert network of Chartered Property and Casualty Underwriters (CPCUs). The benchmark uncovers several model-specific and actionable error modes, including basic tool use errors and a surprising number of insidious hallucinations from one provider. This is part of an ongoing series of benchmarks we are releasing across verticals

Explore how Anthropic Claude + AWS help pharmaceutical companies leverage AI for enhanced data insights and revenue growth.

Discover how enterprises can leverage LLM-as-Judge systems to evaluate generative AI outputs at scale, improve model alignment, reduce costs, and tackle challenges like bias and interpretability.

Ensure your LLMs align with your values and goals using LLM alignment techniques. Learn how to mitigate risks and optimize performance.

Unlock possibilities for your enterprise with LLM distillation. Learn how distilled, task-specific models boost performance and shrink costs.

Learn about large language model (LLM) alignment and how it maximizes the effectiveness of AI outputs for organizations.

Discover how Snorkel AI’s methodical workflow can simplify the evaluation of LLM systems. Achieve better model performance in less time.

Accelerate LLM development with Snorkel Flow and SageMaker. Automate dataset curation, accelerate training, and gain a competitive advantage.

Learn about the obstacles faced by data scientists in LLM evaluation and discover effective strategies for overcoming them.

We aim to help our customers get GenAI into production. In our 2024.R3 release, we’ve delivered some exciting GenAI evaluation results.

Snorkel AI has made building production-ready, high-value enterprise AI applications faster and easier than ever. The 2024.R3 update to our Snorkel Flow AI data development platform streamlines data-centric workflows, from easier-than-ever generative AI evaluation to multi-schema annotation.

Meta’s Llama 3.1 405B, rivals GPT-4o in benchmarks, offering powerful AI capabilities. Despite high costs, it can enhance LLM adoption through fine-tuning, distillation, and as an AI judge.

Meta released Llama 3 405B today, signaling a new era of open source AI. The model is ready to use on Snorkel Flow.

We need more labeled data than ever, so we have explored weak supervision for non-categorical applications—with notable results.

To tackle generative AI use cases, Snorkel AI + AWS launched an accelerator program to address the biggest blocker: unstructured data.

AI alignment ensures that AI systems align with human values, ethics, and policies. Here’s a primer on how developers can build safer AI.

Vision language models demonstrate impressive image classification capabilities, but LLMs can help improve their performance. Learn how.

Snorkel researchers devised a new way to evaluate long context models and address their “lost-in-the-middle” challenges with mediod voting.

See a walkthrough of how Snorkel Flow users build applications with production-grade RAG retrieval components.

Snorkel takes a step on the path to enterprise superalignment with new data development workflows for enterprise alignment

ROBOSHOT acts like a lens on foundation models and improves their zero-shot performance without additional fine-tuning.

Unlock advanced LLM customization with Snorkel Flow’s new release! Explore flexible data integrations, secure controls, and multimodal support to fine-tune language models for enterprise use. Discover how to leverage images and diverse data types for AI-driven insights.

Snorkel Flow’s new FM-first workflow for building document intelligence applications will get you from demo to production faster than ever.

Snorkel’s Paroma Varma and Google’s Ali Arsenjani discus the role of data in the development and implementation of LLMs.

We’re excited to announce Snorkel Custom to help enterprises cross the chasm from flashy chatbot demos to real production AI value.

Snorkel AI placed a model at the top of the AlpacaEval leaderboard. Here’s how we built it, and how it changed AlpacaEval’s metrics.

As Snorkel AI prepares to build better enterprise LLM evaluations, we spoke with Yifan Mail from Stanford’s CRFM HELM project.

A customer wanted an llm system for complex contract question answering tasks. We helped them build it—beating the baseline by 64 points.

Google and Snorkel AI customized PaLM 2 using domain expertise and data development to improve performance by 38 F1 points in a matter of hours.

Microsoft infrastructure facilitates Snorkel AI research experiments, including our recent high rank on the AlpacaEval 2.0 LLM leaderboard.