Knowledge Graph — Coursera Notes › Academic disciplines › Information Technology / Computer Science › Artificial Intelligence › Large Language Models › Foundation Models

Hugging Face Transformers

service · part of Foundation Models

A library providing pretrained models and tools for natural language processing tasks.

It also includes the Trainer and TrainingArguments classes for fine-tuning, and simplifies loading models and adapting them to specific tasks.

BertForSequenceClassification is a BERT model variant with a classification head for sequence classification tasks. It can be loaded with a specified number of labels (num_labels) and used for fine-tuning with PEFT.

model = BertForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=3)

TrainingArguments configures training parameters such as output directory, number of epochs, batch size, evaluation strategy, and learning rate. It is passed to the Trainer to control the fine-tuning process.

training_args = TrainingArguments(
    output_dir='./results',
    num_train_epochs=3,
    per_device_train_batch_size=16,
    evaluation_strategy="epoch",
)

Trainer is a Hugging Face class that handles training and evaluation loops. It takes a model, training arguments, and datasets, and provides methods like train() and evaluate() for fine-tuning and performance assessment.

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_data,
    eval_dataset=val_data,
)
trainer.train()

Inside Hugging Face Transformers (4)

• AutoModelForSeq2SeqLM — A Hugging Face class that automatically loads a sequence-to-sequence model for a given model name.
• AutoTokenizer — A Hugging Face class that automatically loads the tokenizer for a given pretrained model.
• Hugging Face pipeline — A high-level function from the Hugging Face Transformers library that simplifies using models for tasks like text generation, classification, etc.
• Tokenizers — Libraries in HuggingFace Transformers that convert text into tokens for model input.

Connections

• Related to AutoTokenizer
• Related to AutoModelForSeq2SeqLM
• Related to Hugging Face pipeline

This is the text view of an interactive 3D knowledge graph — open this page with JavaScript enabled to explore it visually.