Improving the accuracy of domain-specific tasks

with LLM distillation

The reasoning capabilities of large, state-of-the-art open models such as DeepSeek R1 rival those of popular closed models, but their generalization is rarely required and the compute resources required for low-latency inference often put them out of reach.

However, LLM distillation has proven to be an effective technique for creating small language models (SLMs) which excel at specialized tasks – preserving the reasoning capabilities of large models while at the same time significantly reducing inference costs.

In this webinar, we’ll provide an overview of LLM distillation, explain how it compares with fine-tuning, and introduce the latest techniques for training SLMs using foundation models and knowledge transfer methods.