Tag
AI evaluation systematically measures a model’s performance on tasks. Classically, this applied metrics like accuracy or precision to clear and discrete numerical or categorical targets. Moden evaluation also assesses the output of generative models to ensure they create content within an organization’s standards and guidelines.
All articles on Evaluation
How we built better GenAI with programmatic data development
We used weak supervision to programmatically curate instruction tuning data for open-source LLMs to build a better GenAI.
July 19, 2023
•
Snorkel Flow Summer 2023: faster, easier and more secure
This release eases Snorkel Flow application creation process and tightens the iteration loop. It also upgrades our security certifications.
July 14, 2023
•
How NVIDIA Omniverse bolsters AI with synthetic data
NVIDIA’s Nyla Worker presented “Leveraging Synthetic Data to Train Perception Models Using NVIDIA Omniverse Replicator” in 2022.
July 6, 2023
•
How to build reusable data cleaning pipelines with scikit-learn
State Farm senior data scientist Jason Goldfarb presented “Reusable Data Cleaning Pipelines in Python” at the Future of Data-Centric AI 2022.
July 3, 2023
•
The future of large language models is faster and more robust
Snorkel and affiliated academic labs have been hard at work reducing how computationally expensive large language models are.
June 29, 2023
•
Claypot AI CEO on why you should deploy models the hard way
Claypot AI CEO Chip Huyen presented “Platform for Real-Time Machine Learning” at Snorkel AI’s Future of Data-Centric AI 2022.
June 27, 2023
•
How Grammarly strives for superhuman communication assistance
Grammarly’s Timo Mertens presents “Toward Superhuman Communication Assistance” at Snorkel AI’s The Future of Data-Centric AI Summit in 2022.
June 14, 2023
•
How MLCommons is democratizing data with public datasets
Peter Mattson, Google senior staff engineer and president of MLCommons.org, explained MLCommons at The Future of Data-Centric AI in 2022.
May 31, 2023
•
Stanford professor on data-centric AI for healthcare and medicine
Stanford assistant professor James Zou, presents “Responsible Data-Centric AI for Healthcare and Medicine” at The Future of Data-Centric AI.
May 18, 2023
•
Capital One’s data-centric solutions to banking business challenges
Three experts gave a look into Capital One’s data-centric solutions to solve the bank’s business problems and improve customer experience.
May 12, 2023
•
Dataset cartography: a data science lesson from Capital One
William Huang, senior data scientist at Capital One, discussed “dataset cartography” and its value at the Future of Data-Centric AI 2022.
May 10, 2023
•
Out of distribution blindness: why to fix it and how energy can help
Sharon Li is an assistant professor at the University of Wisconsin-Madison. She presented “Detecting Data Distributional Shift: Challenges and Opportunities” at Snorkel AI’s The Future of Data-Centric AI Summit in 2022. The talk covered a novel approach for handling out-of-distribution objects.
May 3, 2023
•
Debugging data to build better and more fair ML applications
Dr. Ce Zhang is an associate professor in Computer Science at ETH Zürich. He presented “Building Machine Learning Systems for the Era of Data-Centric AI” at Snorkel AI’s The Future of Data-Centric AI event in 2022.
April 28, 2023
•
Harvard professor: DataPerf and AI’s need for data benchmarks
Harvard Professor Vijay Janapa Reddi’s presentation: “DataPerf: Benchmarks for data” from Snorkel AI’s 2022 Future of Data-Centric AI event.
April 25, 2023
•
AMA technique: a trick to build systems with foundation models
Simran Arora is a machine learning researcher at Stanford University. She presented “Ask Me Anything: How are Foundation Models Changing the Way We Build Software” at Snorkel AI’s Foundation Model Virtual Summit 2023.
April 13, 2023
•
Boost foundation model results with linear probing and fine-tuning
Ananya Kumar, Stanford Ph.D. student, explains methods to improve foundation model performance, including linear probing and fine-tuning.
April 5, 2023
•
Adverse drug events: how to spot them with machine learning
Physician notes and other sources of unstructured data can offer insight into drugs’ negative side effects. Here’s how modern machine learning tools can help turn all that data into a useful resource.
March 29, 2023
•
McKinsey QuantumBlack experts: exciting foundation model future
McKinsey’s Carlo Giovine and David Harvey present “Trends in Enterprise ML and the potential impact of Foundation Models” at Snorkel’s Foundation Model Summit.
March 21, 2023
•
HuggingFace research lead on unified foundation models
Amanpreet Singh, Lead Researcher at Hugging Face gave a presentation entitled Towards Unified Foundation Models for Vision and Language Alignment a Snorkel AI’s Foundation Model Summit in January.
March 8, 2023
•
Foundation Models 101: a guide with essential FAQs
Foundation Models (FMs), such as GPT-3 and Stable Diffusion, mark the beginning of a new era in machine learning and artificial intelligence. What are they and how will they impact your business? Find out in our guide.
March 1, 2023
•
Credo AI DS head on operationalizing responsible AI
Credo AI’s head of data science explains at Snorkel’s FDCAI 2022 how his team works to operationalize responsible AI assessment tools.
February 22, 2023
•
Cleanlab CEO shows automatic data-cleansing tools
Cleanlab Co-Founder and CEO Curtis Northcutt presents his company’s automatic, universal and open-source tools to quickly clean data sets.
February 17, 2023
•
NASA ML Lead on its WorldView citizen scientist no-code tool
Anirudh Koul is Machine Learning Lead for the NASA Frontier Development Lab and the Head of Machine Learning Sciences at Pinterest. He presented at Snorkel AI’s 2022 Future of Data Centric AI (FDCAI) Conference.
February 6, 2023
•
Building better datasets with Snorkel Flow error analysis
As machine learning practitioners, few of us would expect the first version of a new model to achieve our objective. We plan for multiple rounds of iteration to address errors and improve performance, and the Snorkel Flow platform provides tools to enable this kind of iteration within the data-centric AI framework.
February 2, 2023
•
Accuracy top concern for Foundation Model adoption—Poll
Most poll respondents at Snorkel AI’s recent Foundation Model Virtual Summit named questionable accuracy as the biggest barrier preventing them from getting organizational value from Foundation Models.
January 31, 2023
•
How Foundation Models bolster programmatic labeling
Snorkel CEO Alex Ratner interviews Mayee Chen about how Liger improves the effectiveness of programmatic labeling through foundation model embeddings.
January 26, 2023
•
Prompting and weak supervision to build better, smaller models
Snorkel AI co-founder and CEO Alex Ratner recently interviewed several Snorkel researchers about their published academic papers. In this video, Alex talks with Ryan Smith, Senior Applied Scientist at Snorkel, about the work he did on using foundation models to build compact, deployable, and effective models.
January 19, 2023
•
FM Summit shows Foundation Model hurdles and potential
Snorkel AI held its Foundation Model Summit Jan 17, bringing together 12 presenters and over 600 attendees at 10 virtual sessions. The event drew registrants from across many sectors, including the tech industry, healthcare, and financial services.
January 18, 2023
•
Ask Me Anything approach bolsters foundation models
Researcher Simran Arora tells Snorkel CEO Alex Ratner how she improved foundation model effectiveness by using “Ask Me Anything”-style questions.
January 4, 2023
•
Combining human and artificial intelligence with human-in-the-loop ML | FDCAI
More components in an ML lifecycle are designed to run on autopilot, but some tasks require human-in-the-loop ML, an active research topic that has seen an increasing number of publications in the last 10 years.
December 28, 2022
•