Resource library
Explore our complete library of resources including blogs, benchmarks, research papers, and more.
Why coding agents need better data, evals, and environments
Closing the Evaluation Gap in Agentic AI
Announcing a $3M commitment to launch Open Benchmarks Grants
Evaluating coding agent capabilities with Terminal-Bench: Snorkel’s role in building the next generation benchmark
Building FinQA: An Open RL Environment for Financial Reasoning Agents
The science of rubric design
Snorkel CEO Alex Ratner chatted with Stanford Professor Percy Liang about evaluation in machine learning and in AI generally.
Learn how enterprises can harness the power of LLMs to deliver tangible results today and see real-world examples of the latest in LLM fine-tuning, distillation, and data development.
Honeypots are a classic cyber-deceptive technique that allows a defender to add false information into the system in an effort to deter/delay/distract potential attackers. However, the effectiveness of honeypots is dependent on their design along with the environment into which they are deployed. In this work, we consider the scenario where there is a collection of honeypots along with a…
LLMs have claimed the spotlight since the debut of ChatGPT, but BERT models quietly handle most enterprise production NLP tasks.
In its first six months, Snorkel Foundry collaborated on high-value projects with notable companies and produced impressive results.
When done right, advanced classification applications cultivate business value and automation, unlock new business lines, and reduce costs.
A brief guide on how financial institutions could use Google Dialogflow with Snorkel Flow to build better chatbots for retail banking
Zero-shot inference is a powerful paradigm that enables the use of large pretrained models for downstream classification tasks without further training. However, these models are vulnerable to inherited biases that can impact their performance. The traditional solution is fine-tuning, but this undermines the key advantage of pretrained models, which is their ability to be used out-of-the-box. We propose ROBOSHOT, a…
Organizations typically train large models individually. This is costly and time-consuming, particularly for large-scale foundation models. Such vertical production is known to be suboptimal. Inspired by this economic insight, we ask whether it is possible to leverage others’ expertise by trading the constituent parts in models, i.e., sets of weights, as if they were market commodities. While recent advances in…
