AI & ML Quick Path

Capstone 8 of 8

Fine-tune a small model with LoRA

Language modelsAdvancedEffort 20–30 h (estimate)

Business requirement

A support team wants a small model that answers in the team’s own short, polite style and returns the same JSON fields every time. You fine-tune a small open model with LoRA on a few hundred examples and prove with a test set that it beats the plain prompted model.

Lessons to finish first

Architecture

  1. Training examples (JSON lines: instruction, input, ideal answer)
  2. Train / validation / test split made before any training
  3. Baseline: the same small model with a good prompt, no fine-tuning
  4. LoRA fine-tuning on a base model (optionally 4-bit compressed: QLoRA)
  5. Validation loss and early stopping
  6. Evaluation on the frozen test set (valid JSON rate, field accuracy, style check)
  7. Merged or adapter model saved with a version

Build steps

  1. Write 300 to 500 synthetic support examples (instruction, input, ideal answer in a fixed JSON shape). Split 80/10/10 into train, validation and test before you look at any result.
  2. Measure the baseline: run the untouched small model with your best prompt on the test set and record valid-JSON rate and field accuracy.
  3. Count the parameters: write down how many weights the base model has and how many LoRA adds (rank r) so you can say what share is trained.
  4. Fine-tune with LoRA. Use a small learning rate with warm-up then decay, gradient clipping and a few epochs. Plot training and validation loss.
  5. Stop early when validation loss stops falling; keep the best checkpoint.
  6. Run the same test set on the tuned model. Compare with the baseline in a table. Read at least 10 wrong answers and group the causes.
  7. Write a one-page note: what improved, what did not, what it cost in time and memory, and whether fine-tuning was worth it compared with only prompting.

Python route and .NET route

Python
Hugging Face transformers with the peft and trl packages (LoRA config with rank 8 to 16, a small base model of about 0.5 to 3 billion parameters, optional bitsandbytes 4-bit loading for QLoRA). Plain PyTorch also works. Needs a GPU for speed; a CPU run on a very small model is possible but slow.
.NET (C#)
Train in Python, then run the merged or exported model from .NET through ONNX Runtime or a local model server (see the local assistant capstone). Evaluation (valid JSON rate, field accuracy) can be written in C# with System.Text.Json.

Example input and output

Input
Email text: "Where is order A-1001? It is late."
Output
{"intent":"delivery","order_id":"A-1001","urgent":false} in the team style

Data

Synthetic examples you write or generate yourself. Never train on customer data you do not have the right to use, and remove personal data first.

Starter file structure

Structure (Python file | .NET project)
lora-finetune/
  data/train.jsonl, val.jsonl, test.jsonl
  baseline.py | Baseline/Program.cs
  train_lora.py
  evaluate.py | Evaluate/Program.cs
  NOTE.md

Tests

  • Test set was split before training and never used for tuning
  • Baseline and tuned model are scored with the same code
  • Valid-JSON rate and field accuracy are reported with the number of test examples
  • Parameter count (total and trained) is reported

How it will be judged

  • Honest comparison with the prompt-only baseline
  • Loss curves plotted and overfitting explained
  • Failure cases read and grouped
  • Cost and memory noted

Security and privacy

Check data licences and the base model licence; remove personal data; do not put secrets in training examples (the model can repeat them).

Definition of done

  • Baseline and tuned scores on the same test set
  • Loss plot and chosen checkpoint
  • One-page decision note

Stretch goals

  • Try ranks 4, 8 and 16 and compare
  • Repeat with 4-bit QLoRA and compare memory
  • Add a small preference-tuning (DPO) round on pairs of good and bad answers

What to show an employer

Comparison table, loss plot and the decision note.

Completion

Saved in this browser only.