DAY 75 / 210

Introduction to Parameter-Efficient Fine-Tuning

Phase 2 begins the shift from prompting to training; establishing PEFT fundamentals today prevents inefficient full fine-tunes later and directly supports StartupTribunal's need for lightweight model adaptation on limited data.

⏱ 45 min target📝 3 quiz Qs

Resources

readingHugging Face
PEFT: State-of-the-art Parameter-Efficient Fine-Tuning
Overview + LoRA section
20 min
readingarXiv
LoRA: Low-Rank Adaptation of Large Language Models
Abstract + Section 2
15 min

Deliverable

Journal entry: 300-word summary of why LoRA beats full fine-tuning for StartupTribunal use-case with one concrete hyperparameter choice

Quiz · 3 questions

1. Which statement about full fine-tuning versus LoRA is false?

Full fine-tuning updates all parametersLoRA adds low-rank matrices while freezing base weightsLoRA always requires more GPU memory than full fine-tuningBoth can be performed on the same base model

2. Name one common misconception when first applying LoRA to a 7B model and the practical consequence.

3. For StartupTribunal's brief-generation task, which single adapter method would you try first and why?

Journal

Time spent (minutes)

Blockers

Commit / PR links (one per line)