Intelligence Per Watt: Measuring Local AI Efficiency
Stanford’s Hazy Research Lab introduces Intelligence per Watt (IPW), a metric for how efficiently AI systems turn energy into accurate answers.
88.7%
Of single-turn chat &
reasoning queries local
LMs answer accurately
5.3x
Improvement in
intelligence-per-watt
from 2023 to 2025
1M
Real-world queries
evaluated across 20+
local models and 8 accelerators
What is Intelligence per Watt?
Intelligence per Watt (IPW) is a metric introduced by Stanford’s Hazy Research Lab to measure the energy efficiency of AI inference: how much accurate “intelligence” a model-accelerator pair delivers for every watt of power it draws. It’s defined simply:
IPW = (mean accuracy across tasks) ÷ (mean power draw during inference)
A higher IPW means a system is converting energy into correct answers more efficiently. The metric is deliberately hardware-and-model agnostic — it captures gains from better model architecture (higher accuracy with fewer parameters) and from better accelerator design (more compute per watt) in a single number, so different model-hardware pairings can be compared on equal footing.
Why it matters
AI demand is exploding. Google Cloud has reported over a 1,000x increase in monthly token processing in the last two years, and the industry’s default response has been to build more data centers. But Hazy Research points out that roughly 77% of real-world ChatGPT usage is routine: writing an email, summarizing a document, answering a factual question. None of that requires frontier-scale reasoning, yet most of it is still routed to massive, centralized models.
History suggests a different path. From 1946 to 2009, computing efficiency doubled roughly every 1.5 years, and that steady efficiency gain — not raw performance — is what let workloads shift from mainframes to personal computers. Hazy Research argues AI inference is at the same inflection point: as local models and local accelerators (laptops, phones, edge devices) get more efficient, a meaningful share of AI demand can move off centralized infrastructure entirely, easing the pressure on power grids and data-center build-outs while preserving frontier compute for the tasks that actually need it.
How the study was built
Hazy Research ran a large-scale empirical study: 20+ state-of-the-art local language models (Qwen3 4B–14B, gpt-oss-20b/120b, Gemma3 4B–12B, IBM Granite 4.0 1B–7B) across 3 local accelerators (2023–2025 hardware, including Apple’s M-series chips) and 5 enterprise-grade accelerators (Nvidia, AMD, and SambaNova) as an efficiency ceiling. The workload was 1 million real-world queries drawn from WildChat (real ChatGPT-style traffic), Natural Reasoning, and standardized benchmarks for knowledge breadth (MMLU-Pro) and expert-level reasoning (SuperGPQA).
Energy was measured directly during single-query inference. NVML counters for Nvidia GPUs, the powermetrics utility for Apple Silicon, sampled every 50ms for higher resolution than prior studies. Accuracy was scored via LLM-as-judge win/tie rates against frontier models (GPT-5, Gemini 2.5 Pro) for open-ended chat queries, and exact match for reasoning and knowledge tasks with ground-truth answers.
Key findings
- Local models are already capable. Local LMs correctly answered 88.7% of single-turn chat and reasoning queries, with accuracy improving 3.1x from 2023 to 2025. Routing each query to the best available local model even beat routing to the best cloud model on 3 of 4 benchmark categories tested.
- Local hardware still trails enterprise hardware on efficiency. Running the same model (Qwen3-32B) on an Apple M4 Max produced 1.5x lower intelligence-per-watt than running it on an Nvidia B200 — meaningful headroom still to close.
- Efficiency is compounding fast. Overall intelligence-per-watt improved 5.3x from 2023 to 2025 (roughly 2.3x per year) split between a 3.1x gain from model-side improvements (architecture, pretraining, post-training, distillation) and a 1.7x gain from hardware advances.
- The study has real scope limits. It covers single-turn mainstream chat and reasoning at batch size 1, not agentic tool use, long-horizon planning, or long-document processing, where local models still lag frontier models by up to 45 percentage points. Hazy Research has flagged this as the next frontier for the work.
Snorkel AI’s role
Intelligence per Watt comes out of Stanford’s Hazy Research Lab, headed by Snorkel AI co-founder Chris Ré. Hazy Research’s initial study focused on general chat and reasoning; Snorkel AI is contributing its own specialized, expert-authored benchmarks, built through Snorkel’s Expert Data-as-a-Service network of domain specialists, to extend IPW into the kind of work enterprises actually run: insurance underwriting, financial analysis, legal review, and PhD-level technical reasoning.
The goal is to answer the questions general-chat benchmarks can’t: how efficiently can a local model handle medical reasoning, regulatory compliance, or underwriting judgment, and can an edge device deliver expert-level performance inside a real power budget? Snorkel is also a listed data and compute supporter of the underlying IPW research and its open profiling harness.
