Knowledge Graph — Coursera Notes › Academic disciplines › Information Technology / Computer Science › Artificial Intelligence › Deep Learning › Model Training

Fine-tuning

concept · part of Model Training

The process of adapting a pre-trained model to a specific task using additional training data.

Key benefits include improved accuracy, reduced data requirements, and leveraging transfer learning.

Pretraining trains a model on a vast corpus (billions of words) to understand language structure, grammar, and semantics generally. Fine-tuning retrains the pretrained model on smaller, task-specific datasets to specialize in tasks like recognizing industry jargon or improving translation accuracy. Without fine-tuning, models lack precision for specialized use cases.

For example, a general LLM can perform sentiment analysis broadly, but fine-tuning for customer support enables it to distinguish nuanced emotions like frustration or excitement in specific queries.

Fine-tuning requires much smaller datasets than pretraining because the model already has a strong language foundation. Quality is paramount: the dataset must be representative of the specific task. For example, fine-tuning for medical terms needs curated clinical data. Careful selection avoids overfitting and biases.

Fine-tuned models can outperform general-purpose models on specific tasks by a substantial margin, providing more accurate and contextually relevant outputs. This is critical for applications like automated customer service. Fine-tuning also allows periodic retraining as new task data is gathered.

Fine-tuning can be performed using techniques like supervised fine-tuning (SFT) with labeled data, or reinforcement learning from human feedback (RLHF) to align model outputs with human preferences. A common code example for fine-tuning a transformer model using Hugging Face's Transformers library is:

from transformers import AutoModelForSequenceClassification, Trainer, TrainingArguments

model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased", num_labels=2)
training_args = TrainingArguments(output_dir="./results", num_train_epochs=3, per_device_train_batch_size=16)
trainer = Trainer(model=model, args=training_args, train_dataset=train_dataset)
trainer.train()

Legal firms fine-tune LLMs to review, summarize, and draft legal contracts. A fine-tuned model can highlight potential legal risks or discrepancies, saving hours of manual review.

Marketers fine-tune LLMs to generate customized content for audience segments, such as personalized emails or targeted social media posts that align with a brand's tone and messaging.

Researchers fine-tune LLMs to summarize large volumes of academic papers or extract key insights, aiding quicker knowledge dissemination.

Organizations tailor models to their needs: a bank fine-tunes a model to detect fraud patterns using transaction data; a healthcare organization fine-tunes to assist in diagnosing diseases from medical records. Fine-tuned LLMs improve decision-making and efficiency in fields requiring accuracy and reliability.

Fine-tuning adapts a pretrained model (e.g., BERT) to a specific task by training it on a labeled dataset. It involves loading a pretrained model, adding a task-specific head (e.g., sequence classification), and training with appropriate hyperparameters like learning rate and batch size.

Pandas DataFrames are converted to Hugging Face Dataset objects using Dataset.from_pandas(). Unnecessary columns (e.g., 'text', 'cleaned_text') are removed before training to avoid errors.

train_dataset = Dataset.from_pandas(train_data)
train_dataset = train_dataset.remove_columns(["text", "cleaned_text"])

DataCollatorWithPadding dynamically pads batches to the longest sequence in each batch, improving efficiency. It is passed to the Trainer's data_collator argument.

data_collator = DataCollatorWithPadding(tokenizer=tokenizer)

TrainingArguments configure hyperparameters like learning_rate (2e-5), batch size (16), number of epochs (3), output directory, logging, and evaluation strategy (per epoch). report_to='none' disables external logging.

training_args = TrainingArguments(
    learning_rate=2e-5,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    num_train_epochs=3,
    output_dir="./results",
    logging_dir="./logs",
    report_to="none",
    save_strategy="epoch",
    evaluation_strategy="epoch",
)

AutoModelForSequenceClassification loads a pretrained model (e.g., 'bert-base-uncased') with a classification head. The num_labels parameter specifies the number of classes (e.g., 3 for positive/negative/neutral).

model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased", num_labels=3)

Required packages include transformers, datasets, scikit-learn, torch, and accelerate. Install via pip before running the code.

!pip install transformers datasets scikit-learn torch accelerate

Required packages include transformers, datasets, scikit-learn, torch, and accelerate. Install via pip before running the code.

!pip install transformers datasets scikit-learn torch accelerate

Task type determines model architecture and dataset preparation. Text classification uses BERT/RoBERTa with labeled balanced datasets. Text generation uses GPT models with aligned input-output pairs. Question answering uses BERT/T5 with question-answer-context format.

Complex tasks (e.g., legal translation, medical records) require domain-specific datasets and models capable of handling specialized language. Fine-tuning a pretrained model is more effective than training from scratch.

Model size should match task size and computational resources. Small tasks (<10k examples, <200 tokens) use BERT-base (110M params) or DistilBERT (66M) with single GPU 8-16GB VRAM. Large tasks (>100k examples, >500 tokens) use GPT-3 (175B) or T5-large (770M) with multiple GPUs/TPUs >=32GB VRAM.

Larger datasets enable learning comprehensive patterns; smaller datasets can still yield strong results if curated and balanced. Augmentation techniques like paraphrasing and backtranslation can improve small datasets.

Use Hugging Face Transformers to load a pretrained model and tokenizer. Example: tokenizer = BertTokenizer.from_pretrained('bert-base-uncased'); model = BertForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=3). The num_labels parameter sets the number of output classes.

from transformers import BertTokenizer, BertForSequenceClassification
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=3)

Datasets can be found on Hugging Face Datasets, Kaggle Datasets, Google Dataset Search, and UCI Machine Learning Repository. Ensure datasets are task-specific, properly labeled, and balanced.

The num_labels parameter in BertForSequenceClassification specifies the number of output classes. For binary classification use 2; for multi-class use the number of categories. Items can be classified multiple times (e.g., spam, not spam, further review).

When comparing traditional fine-tuning, LoRA, and QLoRA, consider performance metrics (accuracy, F1, precision, recall) and resource efficiency (training time, memory usage, computational cost). Traditional fine-tuning: high performance, high resource use. LoRA: reduces memory by fine-tuning low-rank matrices, often without major performance loss. QLoRA: combines quantization with LoRA, further reducing memory while maintaining competitive performance.

Evaluation after fine-tuning is critical to assess generalization, identify overfitting/underfitting, and compare techniques like traditional fine-tuning, LoRA, and QLoRA.

Fine-tuning typically freezes lower layers that capture universal patterns (e.g., edges, curves) while retraining or replacing more task-specific upper layers. This balances leveraging pretrained knowledge with adapting to new data. Techniques like LoRA (Low-Rank Adaptation) and QLoRA (Quantized LoRA) reduce memory usage by fine-tuning low-rank matrices or combining quantization with LoRA, often maintaining competitive performance. Evaluation after fine-tuning assesses generalization and identifies overfitting/underfitting, comparing methods like traditional fine-tuning, LoRA, and QLoRA using metrics such as accuracy, F1, precision, and recall.

Inside Fine-tuning (3)

• Low-Rank Adaptation (LoRA) — Low-Rank Adaptation (LoRA) is a parameter-efficient fine-tuning technique that adapts large pretrained models by applying low-rank matrix modifications to key layers (e.g., attention heads) instead of updating all parameters.
• Parameter-Efficient Fine-Tuning — Parameter-efficient fine-tuning (PEFT) reduces computational cost and memory by updating only a subset of a pretrained model's parameters while freezing the rest.
• QLoRA — Quantized Low-Rank Adaptation (QLoRA) is a fine-tuning technique that combines quantization and low-rank adaptation to reduce memory and computational requirements while maintaining model performance.

Connections

• Uses HuggingFace

This is the text view of an interactive 3D knowledge graph — open this page with JavaScript enabled to explore it visually.