DAY 76 / 210

Fine-Tuning Foundations for Domain Tasks

This first day of phase-2-finetune establishes core concepts before any code changes, ensuring Maku understands trade-offs that will directly shape StartupTribunal's model behavior. It bridges prior general ML exposure to targeted adaptation techniques relevant to legal-domain briefs. The day sets measurable expectations for subsequent implementation days.

⏱ 45 min target📝 2 quiz Qs

Resources

readingHugging Face
Fine-tuning a pretrained model
entire page
25 min
readingarXiv
LoRA: Low-Rank Adaptation of Large Language Models
abstract + section 2
15 min

Deliverable

journal entry (1 page) comparing full fine-tuning vs LoRA for StartupTribunal use-case with one concrete next-step experiment

Quiz · 2 questions

1. Which statement best captures the primary risk of full-parameter fine-tuning on a small legal-brief dataset?

Overfitting to training examples and catastrophic forgetting of general capabilitiesFaster inference at deployment timeLower memory usage during trainingGuaranteed improvement on out-of-domain prompts

2. In one sentence, explain why LoRA can be preferable to full fine-tuning when compute budget is limited.

Journal

Time spent (minutes)

Blockers

Commit / PR links (one per line)