GACS Logo
All engineering diplomas
LLME · 40 hrs · 12 modules

LLM Engineering

From transformer internals to production-grade LLM systems.

Covers tokenization, transformer architecture, fine-tuning, RAG, evaluation, cost control, and deployment of LLM applications at scale.

Capstone project

Ship a Domain-Tuned LLM End-to-End

Take an open base model, fine-tune it on a domain dataset, align it with preference data, evaluate it on a custom benchmark, and serve it under a measured SLO.

30–50 hours over 2–3 weeks · 7 deliverables · rubric out of 100 · passing score 70

Open capstone brief →
  1. 1

    Foundations of LLMs

    Quiz · 10Q · 80% to pass

    History, architecture, transformer basics, scaling laws, and the end-to-end LLM training pipeline.

    Open module
  2. 2

    Tokenization & Text Processing

    Quiz · 10Q · 80% to pass

    BPE, WordPiece, Unigram, vocabulary design, special tokens, and building tokenizers from scratch.

    Pass Module 1 to unlock
  3. 3

    Embeddings & Representation Learning

    Quiz · 10Q · 80% to pass

    Token embeddings, positional encodings, semantic spaces, RoPE/ALiBi, and embedding-level design choices.

    Pass Module 2 to unlock
  4. 4

    Attention Mechanisms

    Quiz · 10Q · 80% to pass

    Self-attention, multi-head attention, QKV math, complexity, causal masking, and efficient variants (FlashAttention, MQA, GQA).

    Pass Module 3 to unlock
  5. 5

    Transformer Architecture

    Quiz · 10Q · 80% to pass

    Block structure, encoder vs decoder, FFNs, normalization, residual streams, and stacking.

    Pass Module 4 to unlock
  6. 6

    Training LLMs

    Quiz · 10Q · 80% to pass

    Loss functions, optimization, data pipelines, mixed precision, distributed training (DDP, FSDP, ZeRO, TP, PP), and checkpointing.

    Pass Module 5 to unlock
  7. 7

    Fine-Tuning & Adaptation

    Quiz · 10Q · 80% to pass

    When to fine-tune, full fine-tuning vs PEFT, LoRA, QLoRA, instruction tuning, dataset design, and serving adapters.

    Pass Module 6 to unlock
  8. 8

    RLHF & Alignment

    Quiz · 10Q · 80% to pass

    Reward models, preference data, PPO, DPO, KTO, IPO, Constitutional AI, and safety alignment.

    Pass Module 7 to unlock
  9. 9

    Evaluation & Benchmarking

    Quiz · 10Q · 80% to pass

    Perplexity, task benchmarks, hallucination metrics, LLM-as-judge, leaderboard contamination, and continuous eval pipelines.

    Pass Module 8 to unlock
  10. 10

    Serving & Deployment

    Quiz · 10Q · 80% to pass

    Inference optimization, quantization, Triton, ONNX, TensorRT, and production concerns.

    Pass Module 9 to unlock
  11. 11

    Scaling & Optimization

    Quiz · 10Q · 80% to pass

    Sharding, ZeRO, DeepSpeed/FSDP, memory optimization, and KV caching.

    Pass Module 10 to unlock
  12. 12

    Capstone Preparation

    Quiz · 10Q · 80% to pass

    Design, build, train, evaluate, and deploy a small transformer LLM end-to-end.

    Pass Module 11 to unlock
Student Handbook →·Instructor Teaching Manual →