Over the past decade, the evolution of AI has been deeply marked by the rise of large language models (LLMs), notably powerful innovations such as OpenAI’s GPT series, Meta’s Llama 2, and Google’s PaLM 2. These models have significantly altered our perception of AI technology, stirring intrigue about their capabilities across various industries.

Yet, like newly recruited employees, these models often have a broad but shallow knowledge base. They can perform general tasks effectively, but they fall short when it comes to executing specific tasks tailored to an organization’s needs. Therefore, there is a clear need for fine-tuning these models to enhance their performance in the enterprise environment.

I recently gave a talk entitled “How to fine-tune and customize LLMs” at Snorkel AI’s Enterprise LLM Virtual Summit. I would recommend that you watch the full talk, but I will summarize the talk’s main points here.

Understanding the limitations of large language models

Imagine hiring a new employee. They have a general understanding of their job but lack company-specific knowledge. They can perform straightforward tasks but need carefully tailored training to handle specialized tasks within the organization.

LLMs fit a similar mold. They can grasp and handle complex concepts, making them capable of executing tasks like information extraction or even powering chatbots. But without the right fine-tuning—aka tailored training—they fail to deliver fully on company-specific or industry-specific demands. Their off-the-shelf performance will rarely meet an enterprise’s needs for production deployment.

But, with a little adaptation, LLM performance can exceed that bar and begin generating enterprise value.

Approaches to adapt LLMs

So how do we fine-tune these LLMs? Let’s explore three primary approaches;

Full fine-tuning

This approach is akin to retraining the entire large language model. However, it requires ample data and computational resources. Think of this as taking a competent person off the street and then sending them to college in preparation for a job. Full fine-tuning involves updating all the model weights, making it a resource-intensive but highly effective fine-tuning method for achieving optimal model performance. This approach is ideal for enterprises with access to large, high-quality datasets and the computational infrastructure to support extensive training. For example, a tech company might use full fine-tuning to adapt an LLM for code generation by training it on proprietary codebases and developer documentation.

Parameter efficient fine-tuning (PEFT)

Think of this as giving your employees additional training. Techniques such as LoRA (Low-Rank Adaptation) make it unnecessary to change the underlying structure or weights of a model. Instead, we just add new, smaller, and more efficient adapters. PEFT is a fine-tuning technique that allows enterprises to adapt pre-existing models without extensive retraining, making it a cost-effective solution for enhancing the model’s performance. For instance, a logistics company might use PEFT to fine-tune an LLM for route optimization by adding lightweight adapters trained on historical shipment data, without altering the core model architecture.

Distillation

This approach trains a newer, smaller model to replicate the actions and decisions of a larger one. It makes the entire process less data-intensive. This is akin to asking a veteran employee to train a new hire on a very specific task. Distillation leverages the knowledge of a pre-trained model to create a fine-tuned model that is both efficient and deployment-ready. For example, a customer service department might use distillation to create a smaller, faster model for handling common queries, reducing latency and operational costs while maintaining high accuracy.

Each of these has its place. What makes sense for your enterprise will vary based on your needs and available resources, but cost-efficient, deployment-ready models will usually involve distillation or PEFT. For enterprises with limited resources, PEFT and distillation offer a practical way to achieve high performance without the overhead of full fine-tuning.

The key to successful enterprise LLM adoption

At the heart of successful AI model training, we find high-quality, task-specific data—your enterprise’s unique linguistic DNA. In a world where AI training models are widely available and supported (thanks to cloud providers and fantastic resources like Hugging Face) your data creates your key differentiator.

Successful training demands that data teams label, curate, and prepare data for the training process. This can be done through a variety of platforms, but Snorkel Flow provides a one-stop solution for better error analysis, targeted labeling, and collaborations with internal experts. It enables users to improve upon pre existing models and collaborate with subject matter experts for better data optimization.

Case study for LLM fine-tuning

Real-world applications of these approaches have yielded tangible results.

We started one customer project by using clever prompting with GPT-4 and achieved an F1 score of 68.4. This improved on the 49.2 F1 score we achieved with zero-shot prompting alone, but fell well short of the performance bar the client set for production deployment.

We utilized GPT-4 in collaboration with Snorkel Flow’s programmatic labeling suite. We provided engineered prompts for various aspects of each document to generate data labels. However, GPT-4, when prompted, still exhibited some errors. To address this, we employed the platform to establish “Labeling Functions” for additional data labeling on the areas with errors. Subsequently, Snorkel Flow merged the GPT-4 labels with programmatically generated labels for the error sections, resulting in a final probabilistic dataset. In this manner, we combined GPT-4 “teaching” a smaller DistillBERT model with an enhanced curriculum, which is the curated data. This fine-tuning process, which included supervised fine-tuning and distillation, significantly improved the model’s performance. By combining the strengths of a large pre-trained model with a smaller, fine-tuned model, we achieved a balance between accuracy and efficiency.

This approach achieved an 84 F1 score—a sure sign of the efficiency of the distillation process.

The future of large language models in your enterprise

Looking ahead, fine-tuning will play an increasingly critical role in AI deployment. Enterprises will likely rely on multiple smaller models (each tailored to handle specific tasks), rather than a single LLM. These specialized smaller models offer better control and efficiency, promising an exciting era of AI-empowered enterprise solutions.

LLMs are incredibly powerful, and they possess the potential to revolutionize AI technology in various industries. They will rarely—if ever—achieve business objectives out of the box. But, with programmatic labeling, fine-tuning, and distillation, LLMs can power robust pipelines that tackle important, high-complexity enterprise use cases with reliable accuracy.

Additional Considerations for Fine-Tuning LLMs

Data Quality and Diversity

The success of fine-tuning heavily depends on the quality and diversity of the training data. Enterprises must ensure that their datasets are representative of the tasks the model will perform, free from biases, and sufficiently large to capture the nuances of the domain. For example, a model trained on biased data may produce skewed results, leading to poor decision-making in critical applications like hiring or loan approvals.

Computational Resources

Fine-tuning LLMs, especially through full fine-tuning, requires significant computational resources. Enterprises must evaluate their infrastructure and consider cloud-based solutions or distributed training frameworks to handle the computational load efficiently. For instance, platforms like AWS SageMaker and Google Vertex AI offer scalable solutions for fine-tuning large models.

Ethical and Regulatory Compliance

As enterprises deploy fine-tuned models, they must ensure compliance with ethical guidelines and regulatory requirements. This includes addressing issues like data privacy, transparency, and fairness in AI decision-making. For example, GDPR compliance is critical for enterprises operating in the European Union, requiring models to handle personal data responsibly.

Learn how to get more value from your PDF documents!

Transforming unstructured data such as text and documents into structured data is crucial for enterprise AI development. On December 17, we’ll hold a webinar that explains how to capture SME domain knowledge and use it to automate and scale PDF classification and information extraction tasks.

Sign up here!