DAY 50 / 210

LoRA Fundamentals for StartupTribunal Fine-Tuning

This first day of phase-2-finetune establishes the core technique that will let Maku replace expensive full-model updates with low-rank adapters inside the existing Maku brief pipeline. Understanding LoRA now prevents later over-engineering of the rate-limiter and API routes when the model must be specialized on tribunal data.

⏱ 50 min target📝 3 quiz Qs

Resources

readingarXiv
LoRA: Low-Rank Adaptation of Large Language Models
Abstract + Sections 1-3
25 min
readingHugging Face
PEFT Library Overview
Quickstart and LoRA configuration sections
15 min

Deliverable

Journal entry saved to app/maku/journal/day50.md that records one concrete LoRA rank and alpha choice justified against the BriefForm submission flow

Quiz · 3 questions

1. Why does LoRA keep the original weight matrix frozen during training?

To reduce GPU memory by only updating low-rank matricesTo increase model sizeTo enable full retraining of all parametersTo bypass the rate limiter

2. In one sentence, state the shape of the two low-rank matrices that replace a weight update ΔW of size d imes k when rank r=8.

3. How might the current rate-limiter in lib/rate-limiter.ts interact with longer fine-tuning runs that use LoRA adapters?

Journal

Time spent (minutes)

Blockers

Commit / PR links (one per line)