Paradigm shift for machine learning

Snorkel’s technology has been used to unlock new ML use cases in healthcare, criminology, and journalism and to power mission critical AI applications for Fortune 500 enterprises such as Chubb, Genentech, Google, and more.

Snorkel is a fundamentally new interface to ML without hand-labeled training data

Mike Tamir

Chief ML Scientist, Head of Machine Learning/AI at Susquehanna International Group and Data Science Faculty, UC Berkeley

For many practical applications, it’s now more productive to hold the neural network architecture fixed, and instead find ways to improve the data.

Andrew Ng

Founder and CEO, Landing AI

Combining weak supervision and reinforcement learning enables Al systems to learn which actions can solve for which tasks. The result is a high-quality dataset and an optimized model.

Xuedong Huang

Technical Fellow and Azure AI CTO

Pioneering technology

Programmatic labeling

NEURIPS

Data Programming: Creating Large Training Sets, Quickly - A. Ratner, et al. 2016

KDD

Interactive Programmatic Labeling for Weak Supervision - B. Cohen-Wang, et al, 2019

ICCV

Scene Graph Prediction With Limited Labels - V. Chen, et al, 2019

SIGMOD

Snorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scale - S. Bach, et al, 2019

NATURE COMMS

Weakly Supervised Classification of Aortic Valve Malformations Using Unlabeled Cardiac MRI Sequences - J. Fries, et al, 2019

CELL PATTERNS

Cross-Modal Data Programming Enables Rapid Medical Machine Learning - J. Dunnmon, et al, 2020

ICML

Fast and Three-Rious: Speed up Weak Supervision With Triplet Methods - D. Fu, et al, 2020

VLDB

Leveraging Organizational Resources to Adapt Models to New Data Modalities - S. Suri, et al, 2020

Weak supervision

SIGMOD

Snorkel MeTaL: Weak Supervision for Multi-Task Learning - A. Ratner, et al, 2018

DEEM @ SIGMOD

Osprey: Weak Supervision of Imbalanced Extraction Problems Without Code - E. Bringer, et al, 2019

VLDB

Snuba: Automating Weak Supervision to Label Training Data - P. Varma and C. Ré, 2019

AAAI

Training Complex Models with Multi-Task Weak Supervision - A. Ratner, et al, 2019

EMNLP

Reference-based Weak Supervision for Answer Sentence Selection using Web Data - V. Krishnamurthy, et al

NeurIPS

WRENCH: A Comprehensive Benchmark for Weak Supervision - J. Zhang, et al

Data-centric AI

SIGMOD

Data Programming With DDLite: Putting Humans in a Different Part of the Loop - H. Ehrenberg, et al, 2016

NEURIPS

Learning to Compose Domain-Specific Transformations for Data Augmentation - A. Ratner, et al, 2017

VLDB

Snorkel: Rapid Training Data Creation With Weak Supervision - Alexander Ratner, Stephen H Bach, Henry Ehrenberg, Jason Fries, Sen Wu, Christopher Ré

NEURIPS

Slice-Based Learning: A Programming Model for Residual Learning - V. Chen, et al, 2019

CIDR

The Role of Massively Multi-Task and Weak Supervision in Software 2.0 - A. Ratner, et al, 2019

TAGLETS: A System for Automatic Semi-Supervised Learning with Auxiliary Data - W. Piriyakulkij, et al

ICLR

Multitask prompted training enables zero-shot task generalization - V. Sanh, et al

ICML

Adversarial Multiclass Learning under Weak Supervision with Performance Guarantees

View more research papers

Snorkel DryBell: Tackling content classification at Google

Snorkel was used to create classifiers of comparable quality to ones trained with tens of thousands of hand-labeled examples, convert non-servable organizational resources to servable models for an average 52% performance improvement, and execute over millions of data points in tens of minutes.
Published at SIGMOD’19
Collaboration with US FDA and Veterans Affairs on text and image data

Snorkel provided 132% average improvements to predictive performance over prior heuristic approaches and came within an average 3.6% of the predictive performance of large hand-curated training sets.
Published at VLDB’19
Snuba: Outperforming automated approaches

In collaborations with users at research labs, Stanford Hospital, and on open-source datasets, Snorkel outperformed other automated approaches like semi-supervised learning by up to 14.4 F1 points.
Published at VLDB’19
Slice-based learning for language and vision tasks

Snorkel improved over baselines in terms of slice-specific and overall performance by up to 19.0 and 4.6 F1 points respectively on applications spanning natural language understanding and computer vision benchmarks as well as production-scale industrial systems.
Read Research Paper
Cross-modal data programming

Snorkel yielded models that on average perform within 1.75 points and 10.3 ROC-AUC of those supervised with physician-years and -months of hand labeling respectively while using only person-days of developer time and clinician work—a time saving of 96%.
Published at Cell Patterns’20

Snorkel DryBell: Tackling content classification at Google

Snorkel was used to create classifiers of comparable quality to ones trained with tens of thousands of hand-labeled examples, convert non-servable organizational resources to servable models for an average 52% performance improvement, and execute over millions of data points in tens of minutes.

Published at SIGMOD’19

Collaboration with US FDA and Veterans Affairs on text and image data

Snorkel provided 132% average improvements to predictive performance over prior heuristic approaches and came within an average 3.6% of the predictive performance of large hand-curated training sets.

Published at VLDB’19

Snuba: Outperforming automated approaches

In collaborations with users at research labs, Stanford Hospital, and on open-source datasets, Snorkel outperformed other automated approaches like semi-supervised learning by up to 14.4 F1 points.

Published at VLDB’19

Slice-based learning for  language and vision tasks

Snorkel improved over baselines in terms of slice-specific and overall performance by up to 19.0 and 4.6 F1 points respectively on applications spanning natural language understanding and computer vision benchmarks as well as production-scale industrial systems.

Read Research Paper

Cross-modal data programming

Snorkel yielded models that on average perform within 1.75 points and 10.3 ROC-AUC of those supervised with physician-years and -months of hand labeling respectively while using only person-days of developer time and clinician work—a time saving of 96%.

Read Research Paper

Research at Snorkel AI

Our research team works closely with partners in academia and industry to make data-centric AI ubiquitous. The Snorkel AI team regularly publishes in academic journals, contributes to open source projects, applies research to the Snorkel Flow platform, and is faculty at the world's leading educational institutions.