Resource library
Explore our complete library of resources including blogs, benchmarks, research papers, and more.
Why coding agents need better data, evals, and environments
Closing the Evaluation Gap in Agentic AI
Announcing a $3M commitment to launch Open Benchmarks Grants
Evaluating coding agent capabilities with Terminal-Bench: Snorkel’s role in building the next generation benchmark
Benchtalks #2: The future of coding benchmarks
Featuring John Yang (SWE-bench, ProgramBench)
Building FinQA: An Open RL Environment for Financial Reasoning Agents
A challenge in training discriminative models like neural networks is obtaining enough labeled training data. Recent approaches use generative models to combine weak supervision sources, like user-defined heuristics or knowledge bases, to label training data. Prior work has explored learning accuracies for these sources even without ground truth labels, but they assume that a single accuracy parameter is sufficient to…
This paper presents a flexible interface layer to write labeling functions based on experience.
A paradigm for labeling training datasets programmatically rather than by hand.
Introducing DDLite, an interactive development framework for data programming.
