Tag

GenAI

All articles on GenAI

Image
Why GenAI evaluation requires SME-in-the-loop for validation and trust
It’s critical enterprises can trust and rely on GenAI evaluation results, and for that, SME-in-the-loop workflows are needed. In my first blog post on enterprise GenAI evaluation, I discussed the importance of specialized evaluators as a scalable proxy for SMEs. It simply isn’t practical to task SMEs with performing manual evaluations – it can take weeks if not longer, unnecessarily
March 20, 2025
Shane Johnson
Performance of different models on five different benchmarks.
Research spotlight: is long chain-of-thought structure all that matters when it comes to LLM reasoning distillation?
We’re taking a look at the research paper, LLMs can easily learn to reason from demonstration (Li et al., 2025), in this week’s community research spotlight. It focuses on how the structure of reasoning traces impacts distillation from models such as DeepSeek R1. What’s the big idea regarding LLM reasoning distillation? The reasoning capabilities of powerful models such as DeepSeek
March 19, 2025
Shane Johnson
Image
Why enterprise GenAI evaluation requires fine-grained metrics to be insightful
GenAI needs fine-grained evaluation for AI teams to gain actionable insights.
March 18, 2025
Shane Johnson
Image
What is specialized GenAI evaluation, and why is it so critical to enterprise AI?
Specialized GenAI evaluation ensures AI assistants meet business requirements, SME expertise, and industry regulations—critical for production-ready AI.
March 5, 2025
Shane Johnson