Research

Accuracy top concern for Foundation Model adoption—Poll

January 31, 2023
3 min read

Most poll respondents at Snorkel AI’s recent Foundation Model Virtual Summit named questionable accuracy as the biggest barrier preventing them from getting organizational value from Foundation Models.

The January 17 Summit brought together 12 presenters and more than 600 attendees at 10 virtual sessions. Between presentations, we polled attendees on how they expect Foundation Models to fit into their organizations. The results point to broad enthusiasm for Foundation Models (FMs) and Large Language Models (LLMs). But hesitations endure.

Attendees ranked concerns about governance and cost closely behind quality. Notably, just 8% of respondents said they expected “poor use case fit” to block them from using FMs, which suggests that almost all respondents expect to make use of FMs at some point, and tracks closely with other poll results from the same event.

Poll results: top uses and challenges of FMs

Foundation Models will likely provide their greatest value through classification and extraction tasks, according to nearly half of the respondents to our poll—though realizing that value may have to wait. Enterprises generally require a high rate of accuracy to deploy new classification or extraction tools into production, and the results of our first question show that respondents aren’t sure FMs are ready to clear that bar.

We also asked attendees to estimate the timeline by which they expected their organization to deploy its first production use of FMs or LLMs. Of those who responded, only 6% said they were unlikely to use Foundation Models in their business. That mirrors the 8% who named “poor use-case fit” as the reason FMs may not yield value for their organization, but there is some nuance between these questions.

Deploying a new resource doesn’t necessarily mean that the organization will get value out of it. Many experiments fail, but it appears that most of our attendees are eager to experiment. Nearly two-thirds of respondents expected their organization to launch its first production use of FMs in the next year—if it hasn’t already.

Final thoughts

The above results are subject to meaningful error due to our small sample size and significant sampling bias; the respondents all chose to attend a half-day educational session about Foundation Models and answer our optional questions.

However, given that our attendees spanned many industries, including technology, healthcare, and financial services, we think these results—flawed as they are—indicate substantial interest in this new era of artificial intelligence and machine learning.

Like our attendees, we at Snorkel are excited about Foundation Models. Our researchers have published papers on how to get the greatest value from FMs, and our engineers have integrated FM-enabled features into the Snorkel Flow platform. 

To see our new Foundation Model features in action, register now for a Snorkel Flow live demo on February 16, 2023.

Share this article
Image
Matt Casey
Data Science Content Lead

Matt Casey leads content production at Snorkel AI. In prior roles, Matt built machine learning models and data pipelines as a data scientist. As a journalist, he produced written and audio content for outlets including The Boston Globe and NPR affiliates.

Recommended articles

View all articles
Image
Train-to-Test (T²) Scaling Laws: Why Reasoning Models Should Be Overtrained
Nicholas Roberts presents Train-to-Test (T²) scaling laws, which jointly optimize model size, training tokens, and test-time samples—and show why reasoning models should be overtrained well beyond Chinchilla-optimal ratios.
August 18, 2026
Snorkel Team
Image
Milestone-Based Evaluation and Training for Long-Horizon AI Agents
Long-horizon agents operate across many dependent states and transitions, often spanning multiple tools, environments, and periods of external feedback. The difficulty comes from preserving coherent progress as earlier decisions constrain later actions. A single workflow may involve researching evidence, changing files or records, waiting for external responses, revising plans, validating intermediate results, and returning to earlier systems with new information.
August 5, 2026
Zhengyang (Jason) Qi
Image
Enterprise environments and training AI agents for real-world workflows
Most agent benchmarks still evaluate a thin slice of the job. The agent receives a task, produces an answer, gets scored, and the episode ends. Enterprise workflows work differently. An underwriting agent may need to read policy documents, inspect customer records, call internal tools, ask a simulated user for missing information, update state, and follow approval rules. A correct final
August 3, 2026
Chris Glaze
,
Ramya Ramakrishnan
,
Joe Licata
Image

Join our newsletter

For expert advice, the latest research, and exclusive events.
By submitting this form, I acknowledge I will receive email updates from Snorkel AI, and I agree to the Terms of Use and acknowledge that my information will be used in accordance with the Privacy Policy.