Search result for:
LLM evaluation in enterprise applications: a new era in ML
Learn about the obstacles faced by data scientists in LLM evaluation and discover effective strategies for overcoming them.
Our picks
Call center AI for customer experience management: a case study
How one large financial institution used call center AI to inform customer experience management with real-time data.
How a global financial services company built a specialized AI copilot accurate enough for production
Learn how Snorkel, Databricks, and AWS enabled the team to build and deploy small, specialized, and highly accurate models which met their AI production requirements and strategic goals.
Download the Gartner® Quick Answer: Building an AI-Ready Data Management Practice report!
Latest posts
How LLM evaluation drives better models in Snorkel Flow
Discover how Snorkel AI’s methodical workflow can simplify the evaluation of LLM systems. Achieve better model performance in less time.
Unlock proprietary data with Snorkel Flow and Amazon SageMaker
Accelerate LLM development with Snorkel Flow and SageMaker. Automate dataset curation, accelerate training, and gain a competitive advantage.
LLM evaluation in enterprise applications: a new era in ML
Learn about the obstacles faced by data scientists in LLM evaluation and discover effective strategies for overcoming them.
Snorkel AI joins the AWS ISV Accelerate Program and launches Snorkel Flow Availability in AWS Marketplace
Snorkel AI and AWS are partnering to help enterprises build, deploy, and evaluate custom, production-ready AI models. Learn how.
Latest videos
GenAI Evaluation: From Specialized Metrics to Actionable Insights
To Evaluate LLMs, Enterprises Need Specialized, Fine-Grained, and Actionable Insights
LLM Evaluation: How to Evaluate LLMs for Better Enterprise Performance
How to Evaluate LLM Performance for Domain-Specific Use Cases
Understand the basics of LLM training in under four minutes!
RAG Optimization: A Practical Overview for Improving Retrieval Augmented Generation
Applied AI
RAG: LLM performance boost with retrieval-augmented generation
Retrieval-augmented generation (RAG) enables LLMs to produce more accurate responses by finding and injecting relevant context. Learn how.
Snorkel AI signs strategic collaboration agreement with AWS to help enterprises cross the demo-to-production chasm
To tackle generative AI use cases, Snorkel AI + AWS launched an accelerator program to address the biggest blocker: unstructured data.
AI alignment made simple: innovative solutions for businesses
AI alignment ensures that AI systems align with human values, ethics, and policies. Here’s a primer on how developers can build safer AI.
Walking safely before building flying saucer seatbelts: introducing Enterprise Alignment
Snorkel takes a step on the path to enterprise superalignment with new data development workflows for enterprise alignment
Customers
Call center AI for customer experience management: a case study
How one large financial institution used call center AI to inform customer experience management with real-time data.
How we achieved 89% accuracy on contract question answering
A customer wanted an llm system for complex contract question answering tasks. We helped them build it—beating the baseline by 64 points.
Content filtering breakthrough: Snorkel client reaches 96% recall in 3 days
Snorkel AI helped a client solve the challenge of social media content filtering quickly and sustainably. Here’s how.
First cohort of Snorkel GenAI customers sees gains up to 54 points
In its first six months, Snorkel Foundry collaborated on high-value projects with notable companies and produced impressive results.
Data development
LLM evaluation in enterprise applications: a new era in ML
Learn about the obstacles faced by data scientists in LLM evaluation and discover effective strategies for overcoming them.
AI data development: a guide for data science projects
What is AI data development? AI data development includes any action taken to convert raw information into a format useful to AI.
How a global financial services company built a specialized AI copilot accurate enough for production
Learn how Snorkel, Databricks, and AWS enabled the team to build and deploy small, specialized, and highly accurate models which met their AI production requirements and strategic goals.
Task Me Anything: innovating multimodal model benchmarks
“Task Me Anything” empowers data scientists to generate bespoke benchmarks to assess and choose the right multimodal model for their needs.
Product
How LLM evaluation drives better models in Snorkel Flow
Discover how Snorkel AI’s methodical workflow can simplify the evaluation of LLM systems. Achieve better model performance in less time.
Unlock proprietary data with Snorkel Flow and Amazon SageMaker
Accelerate LLM development with Snorkel Flow and SageMaker. Automate dataset curation, accelerate training, and gain a competitive advantage.
Snorkel AI joins the AWS ISV Accelerate Program and launches Snorkel Flow Availability in AWS Marketplace
Snorkel AI and AWS are partnering to help enterprises build, deploy, and evaluate custom, production-ready AI models. Learn how.
SnorkelCon 2024: Inaugural Snorkel AI user conference gathers leaders from 30+ Fortune 500 companies
Discover highlights of Snorkel AI’s first annual SnorkelCon user conference. Explore Snorkel’s programmatic AI data development achievements.
Research
Long context models in the enterprise: benchmarks and beyond
Snorkel researchers devised a new way to evaluate long context models and address their “lost-in-the-middle” challenges with mediod voting.
How ROBOSHOT boosts zero-shot foundation model performance
ROBOSHOT acts like a lens on foundation models and improves their zero-shot performance without additional fine-tuning.
Snorkel teams with Microsoft to showcase new AI research at NVIDIA GTC
Microsoft infrastructure facilitates Snorkel AI research experiments, including our recent high rank on the AlpacaEval 2.0 LLM leaderboard.
How Skill-it! enables faster, better LLM training
Humans learn tasks better when taught in a logical order. So do LLMs. Researchers developed a way to exploit this tendency called “Skill-it!”
Trending stories
1
Snorkel AI has made building production-ready, high-value enterprise AI applications faster and easier than ever. The 2024.R3 update to our Snorkel Flow AI data development platform streamlines data-centric workflows, from easier-than-ever generative AI evaluation to multi-schema annotation.
2
We started the Snorkel project at the Stanford AI lab in 2015 around two core hypotheses:
3
Machine Learning Whiteboard (MLW) Open-source Series Today, Ryan Smith, machine learning research engineer at Snorkel AI, talks about prompting methods with language models and some applications they have with weak supervision. In this talk, we’re essentially going to be using this paper as a template—this paper is a great survey over some methods in prompting from the last few years…
4
The Future of Data-Centric AI Talk Series Background Roshni Malani received her PhD in Software Engineering from the University of California, San Diego, and has previously worked on Siri at Apple and as a founding engineer for Google Photos. She gave a presentation at the Future of Data-Centric AI virtual conference in September 2021. Her presentation is below, lightly edited…
5
We’re excited to announce the Q4 2021 LTS release of Snorkel Flow, our data-centric AI development platform powered by programmatic labeling. This latest release introduces a number of new product capabilities and enhancements, from a streamlined programmatic data development interface, to enhanced auto-suggest for labeling functions, to new machine learning capabilities like AutoML, to significant performance enhancements for PDF data…
6
Microsoft infrastructure facilitates Snorkel AI research experiments, including our recent high rank on the AlpacaEval 2.0 LLM leaderboard.
Latest posts
How LLM evaluation drives better models in Snorkel Flow
Discover how Snorkel AI’s methodical workflow can simplify the evaluation of LLM systems. Achieve better model performance in less time.
Unlock proprietary data with Snorkel Flow and Amazon SageMaker
Accelerate LLM development with Snorkel Flow and SageMaker. Automate dataset curation, accelerate training, and gain a competitive advantage.
LLM evaluation in enterprise applications: a new era in ML
Learn about the obstacles faced by data scientists in LLM evaluation and discover effective strategies for overcoming them.
Snorkel AI joins the AWS ISV Accelerate Program and launches Snorkel Flow Availability in AWS Marketplace
Snorkel AI and AWS are partnering to help enterprises build, deploy, and evaluate custom, production-ready AI models. Learn how.
AI data development: a guide for data science projects
What is AI data development? AI data development includes any action taken to convert raw information into a format useful to AI.
SnorkelCon 2024: Inaugural Snorkel AI user conference gathers leaders from 30+ Fortune 500 companies
Discover highlights of Snorkel AI’s first annual SnorkelCon user conference. Explore Snorkel’s programmatic AI data development achievements.
Explore the new GenAI Evaluation Suite: Snorkel 2024.R3
We aim to help our customers get GenAI into production. In our 2024.R3 release, we’ve delivered some exciting GenAI evaluation results.
Snorkel Flow 2024.R3: Supercharge your AI development with enhanced data-centric workflows
Snorkel AI has made building production-ready, high-value enterprise AI applications faster and easier than ever. The 2024.R3 update to our Snorkel Flow AI data development platform streamlines data-centric workflows, from easier-than-ever generative AI evaluation to multi-schema annotation.
New NLP features in Snorkel Flow 2024.R3
Discover new NLP features in Snorkel Flow\’s 2024.R3 release, including named entity recognition for PDFs + advanced sequence tagging tools.
Enterprise data compliance and security review: Snorkel Flow 2024.R3
Discover the latest enterprise readiness features for Snorkel Flow. Configure safeguards for data compliance and security.
How a global financial services company built a specialized AI copilot accurate enough for production
Learn how Snorkel, Databricks, and AWS enabled the team to build and deploy small, specialized, and highly accurate models which met their AI production requirements and strategic goals.
Task Me Anything: innovating multimodal model benchmarks
“Task Me Anything” empowers data scientists to generate bespoke benchmarks to assess and choose the right multimodal model for their needs.
Alfred: Data labeling with foundation models and weak supervision
Introducing Alfred: an open-source tool for combining foundation models with weak supervision for faster development of academic data sets.
RAG: LLM performance boost with retrieval-augmented generation
Retrieval-augmented generation (RAG) enables LLMs to produce more accurate responses by finding and injecting relevant context. Learn how.
Call center AI for customer experience management: a case study
How one large financial institution used call center AI to inform customer experience management with real-time data.
New GenAI features, data annotation: Snorkel Flow 2024.R2
This release features new GenAI tools and Multi-Schema Annotation, as well as new enterprise security tools and an updated home page.
How data slices transform enterprise LLM evaluation
Enterprises must evaluate LLM performance for production deployment. Custom, automated eval + data slices present the best path to production.
Meta’s Llama 3.1 405B is the new Mr. Miyagi, now what?
Meta’s Llama 3.1 405B, rivals GPT-4o in benchmarks, offering powerful AI capabilities. Despite high costs, it can enhance LLM adoption through fine-tuning, distillation, and as an AI judge.
Meta’s new Llama 3.1 models are here! Are you ready for it?
Meta released Llama 3 405B today, signaling a new era of open source AI. The model is ready to use on Snorkel Flow.
Data-centric AI with Snorkel and MinIO
High-performing AI systems require more than a well-designed model. They also require properly constructed training and testing data.
Weak supervision for non-categorical applications + superalignment
We need more labeled data than ever, so we have explored weak supervision for non-categorical applications—with notable results.
Snorkel AI signs strategic collaboration agreement with AWS to help enterprises cross the demo-to-production chasm
To tackle generative AI use cases, Snorkel AI + AWS launched an accelerator program to address the biggest blocker: unstructured data.
AI alignment made simple: innovative solutions for businesses
AI alignment ensures that AI systems align with human values, ethics, and policies. Here’s a primer on how developers can build safer AI.
How does the Snorkel Flow label model work?
The Snorkel Flow label model plays an instrumental role in driving the enterprise value we create. Here’s a peek at how it works.
Vision language models: how LLMs boost image classification
Vision language models demonstrate impressive image classification capabilities, but LLMs can help improve their performance. Learn how.
Long context models in the enterprise: benchmarks and beyond
Snorkel researchers devised a new way to evaluate long context models and address their “lost-in-the-middle” challenges with mediod voting.
How to build production-grade RAG retrieval with Snorkel Flow
See a walkthrough of how Snorkel Flow users build applications with production-grade RAG retrieval components.
How Bonito helps fine-tune specialized LLMs faster than ever
Fine-tuning specialized LLMs demands a lot of time and cost We developed Bonito to make this process faster, cheaper, and easier.
Walking safely before building flying saucer seatbelts: introducing Enterprise Alignment
Snorkel takes a step on the path to enterprise superalignment with new data development workflows for enterprise alignment
Role-based access controls in Snorkel Flow secure enterprise data
Snorkel Flow’s 2024.R1 release includes new role-based access control tools to further safeguard valuable enterprise data.
Join our newsletter for expert advice, the latest research, and exclusive events.
By submitting this form, I acknowledge I will receive email updates from Snorkel AI, and I agree to the Terms of Use and acknowledge that my information will be used in accordance with the Privacy Policy.