Skip to main content
GACS will never ask for your seed phrase, private keys, or payment. free and ad-free.
GACS.app — Global Anti-Crime & Safety
Sign up / Log in
Back to lessonLab · Module 7 / 12
Lab

Fine-Tuning & Adaptation

Hands-on lab with starter code you can copy + run, and guided steps to produce the artifact for this module.

Setup

  1. Open a fresh terminal in a new folder, e.g. mkdir lab-7 && cd lab-7.
  2. Create a Python 3.11 venv: python -m venv .venv && source .venv/bin/activate.
  3. Install the dependencies noted on each starter snippet (the small pip install … line above the code).
  4. Copy each starter into a file with the suggested name, then run it and capture the output.

Starter code

01 · LoRA fine-tune with PEFT
pip install peft trl datasets bitsandbytes
save as: 07_01_lab.py
python
from peft import LoraConfig, get_peft_model
from transformers import AutoModelForCausalLM, AutoTokenizer
from trl import SFTTrainer
from datasets import load_dataset

base = "Qwen/Qwen2.5-1.5B"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, load_in_4bit=True, device_map="auto")
model = get_peft_model(model, LoraConfig(
    r=16, lora_alpha=32, lora_dropout=0.05,
    target_modules=["q_proj","k_proj","v_proj","o_proj"],
    bias="none", task_type="CAUSAL_LM",
))

ds = load_dataset("trl-lib/Capybara", split="train[:1%]")
SFTTrainer(model=model, tokenizer=tok, train_dataset=ds,
           dataset_text_field="messages", max_seq_length=2048).train()
Run with: python 07_01_lab.py
02 · Merge LoRA adapters back into base weights
save as: 07_02_lab.py
python
from peft import PeftModel
merged = PeftModel.from_pretrained(base_model, "out/checkpoint-100").merge_and_unload()
merged.save_pretrained("out/merged")
Run with: python 07_02_lab.py

Guided steps

Work through each step in order. Stop when your output matches the expected artifact — then push your work to your lab folder.

  1. 1
    LoRA fine-tune on your own data

    Do — Curate 200 (instruction, response) pairs in your domain. LoRA-tune Qwen2.5-1.5B for 1 epoch on a single GPU.

    Expected artifact — Adapter directory <50MB; before/after qualitative comparison on 5 held-out prompts.

    Run free:
  2. 2
    Adapter merge + serve

    Do — Merge your LoRA adapter into the base model and load the merged model. Generate the same 5 prompts.

    Expected artifact — Merged-model outputs match adapter-loaded outputs token-for-token at temperature 0.

    Run free:
  3. 3
    Catastrophic forgetting check

    Do — Run MMLU on the base model and on your fine-tuned model. Report the delta.

    Expected artifact — A 2-row table; expect a small regression — discuss mitigation in your report.

    Run free:

Authoritative sources

Independent primary sources used to check and corroborate the guidance on this page.