LLM Engineering

You can complete every lab in this diploma — including fine-tuning and distributed-training exercises — on free GPU notebooks. Set up once and keep these tabs bookmarked.

1. 1. Use Google Colab for the single-GPU labs

   Colab's free tier gives you a T4 GPU (~15 GB VRAM) for several hours per day. Enough for tokenizer training, attention-from-scratch, LoRA/QLoRA fine-tuning of 1B–3B models, and most evaluation labs. Switch to `Runtime → Change runtime type → T4 GPU` before running.

   Open Google Colab →
2. 2. Use Kaggle Notebooks for longer or larger jobs

   Kaggle gives ~30 hours/week of free GPU (T4 x2 or P100). Better for the fine-tuning labs that run more than 1 hour. Same code as Colab — only the notebook host changes.

   Open Kaggle Notebooks →
3. 3. Substitute a smaller model for 7B+ labs

   Every lab that calls for Mistral-7B or Llama-3-8B has a documented `SMALL_MODEL` swap to Qwen2.5-0.5B, TinyLlama-1.1B, or Llama-3.2-1B. The engineering lessons (LoRA target modules, FlashAttention wiring, RLHF loop) are identical — only the loss curves and final quality differ. The diploma's rubric grades on the correctness of the engineering, not the model size.

4. 4. Fake the multi-GPU labs with gradient accumulation

   For FSDP / ZeRO / distributed-training modules, free tiers only give one GPU. Each lab includes a `single_gpu_simulation` flag that uses gradient accumulation + CPU offload to reproduce the same numerical behavior on one T4. You'll learn the same sharding mental model without paying for 2x A100s.

5. 5. Use a free inference API for the deployment + RAG modules

   For modules on serving, evaluation, and RAG you don't need to host a model yourself. Use Groq (free Llama 3.1/3.3), Together AI's free tier, or HuggingFace Inference API. Every lab supports an OpenAI-compatible base URL.

   Create a free Groq API key →
6. 6. Optional: rent a real GPU for the capstone only

   The end-to-end capstone is the one place where renting an A10 / A100 for 1–2 hours feels worth it. RunPod, Vast.ai, and Lambda all let you spin one up for $0.40–$1.50/hour. Budget $5–10 total. This is optional — the capstone rubric also accepts a small-model variant.

• Colab disconnects after ~90 minutes idle. Save checkpoints to Google Drive at the end of each training loop — every lab notebook already does this.
• If a lab needs more than 15 GB VRAM with the small-model swap, that's a bug — file it via the support link and we'll add a smaller variant.