Knowledge Graph — Coursera Notes › Academic disciplines › Information Technology / Computer Science › Artificial Intelligence › Deep Learning › Model Training › Fine-tuning

Low-Rank Adaptation (LoRA)

concept · part of Fine-tuning

Low-Rank Adaptation (LoRA) is a parameter-efficient fine-tuning technique that adapts large pretrained models by applying low-rank matrix modifications to key layers (e.g., attention heads) instead of updating all parameters. This reduces memory usage, computational cost, and training time, making it suitable for resource-constrained environments.

QLoRA applies LoRA after quantization.

This is the text view of an interactive 3D knowledge graph — open this page with JavaScript enabled to explore it visually.