LLM Engineering & Applied AI Architecture — Capstone Exam

What this exam covers

100 questions drawn from the 24 chapters of the Designing & Building Large Language Models — Study Manual. Pass mark 80%. Read the study book first. Then try the free practice exam with answers.

1. Chapter 0: Orientation — Start Here: How to Succeed in This Program

   Welcome, what's included, technical requirements, time commitment, how to study, how to pass the exam, how to complete the capstone, and how to get help.

   • Welcome
   • What's included
   • Technical requirements
   • Time commitment
   • How to study
   • How to pass the exam
   • How to complete the capstone
   • How to get help

2. Chapter 1: Foundations of Large Language Models

   How modern LLMs work end to end: tokens, embeddings, transformer blocks, autoregressive generation, and scaling.

   • What an LLM actually is
   • Why scale matters
   • Core vocabulary

3. Chapter 2: Tokenization & Embeddings

   How raw text becomes numbers and why this layer silently controls cost, quality and multilingual performance.

   • Subword tokenization
   • Embeddings as meaning
   • Operational impact

4. Chapter 3: Transformer Architecture in Depth

   Self-attention, multi-head attention, residuals, LayerNorm and positional encoding — the actual machinery.

   • Self-attention
   • Multi-head & FFN
   • Residuals, LayerNorm, positions

5. Chapter 4: Pre-Training at Scale

   Distributed training, optimizers, mixed precision and the engineering reality of multi-GPU runs.

   • Parallelism strategies
   • Optimizer and precision
   • Stability

6. Chapter 5: Fine-Tuning Techniques for LLMs

   Why fine-tuning exists, the five major approaches (Full, LoRA, QLoRA, PEFT, Instruction Tuning), domain adaptation, safety alignment, and how to build and evaluate a real pipeline.

   • 5.0 — Why fine-tuning exists
   • 5.1 — What fine-tuning actually does
   • 5.2 — The five major types
   • 5.3 — Full fine-tuning
   • 5.4 — LoRA (Low-Rank Adaptation)
   • 5.5 — QLoRA
   • 5.6 — PEFT (Parameter-Efficient Fine-Tuning)
   • 5.7 — Instruction tuning
   • 5.8 — Domain adaptation
   • 5.9 — Safety alignment
   • 5.10 — The fine-tuning pipeline
   • 5.11 — Evaluation

7. Chapter 6: Evaluation & Benchmarks

   Designing trustworthy evals, avoiding contamination, and reading benchmark results without lying to yourself.

   • What good evals look like
   • Standard benchmarks
   • Contamination & gaming

8. Chapter 7: Deployment & Inference Optimization

   Turning a trained LLM into a real product: pipelines, GPU vs CPU, quantization, distillation, serving architectures, caching, KV-cache, streaming, batching, and cost optimization.

   • 7.0 — Why deployment matters
   • 7.1 — What inference is
   • 7.2 — The inference pipeline
   • 7.3 — GPU vs CPU inference
   • 7.4 — Quantization
   • 7.5 — Distillation
   • 7.6 — Model serving architectures
   • 7.7 — Load balancing
   • 7.8 — Caching
   • 7.9 — KV-cache explained
   • 7.10 — Token streaming
   • 7.11 — Batch inference
   • 7.12 — Serverless LLMs
   • 7.13 — Inference optimization techniques
   • 7.14 — Cost optimization
   • 7.15 — Monitoring & logging
   • 7.16 — Failure modes in deployment
   • 7.17 — Deployment readiness checklist

9. Chapter 8: Building Your Own LLM API

   Turning an LLM into a real product: gateway, auth, rate limiting, safety filters, load balancing, streaming, logging, monitoring, versioning, scaling, and security.

   • 8.0 — Why every LLM needs an API
   • 8.1 — What an LLM API is
   • 8.2 — Architecture overview
   • 8.3 — API gateway
   • 8.4 — Authentication
   • 8.5 — Rate limiting
   • 8.6 — Safety filters
   • 8.7 — Load balancing
   • 8.8 — Inference server
   • 8.9 — Tokenization layer
   • 8.10 — Sampling methods
   • 8.11 — Streaming responses
   • 8.12 — Logging
   • 8.13 — Monitoring
   • 8.14 — Error handling
   • 8.15 — Versioning
   • 8.16 — Scaling strategies
   • 8.17 — Security

10. Chapter 9: Building a Chatbot Interface

    Turning an LLM API into a real product people can use: UI/UX, sessions, memory, safety, RAG, streaming, formatting, error handling, multi-turn reasoning, analytics, deployment, and security.

    • 9.0 — Why chatbots matter
    • 9.1 — What makes a good chatbot
    • 9.2 — Architecture overview
    • 9.3 — Frontend UI design
    • 9.4 — Message handling
    • 9.5 — Session management
    • 9.6 — Memory systems
    • 9.7 — Safety filters
    • 9.8 — RAG integration
    • 9.9 — Streaming responses
    • 9.10 — Response formatting
    • 9.11 — Error handling
    • 9.12 — Multi-turn reasoning
    • 9.13 — Analytics & logging
    • 9.14 — Chatbot deployment
    • 9.15 — Chatbot security
    • 9.16 — UX principles

11. Chapter 10: Retrieval-Augmented Generation (RAG)

    Connect the LLM to external knowledge so it stops hallucinating: embeddings, vector DBs, similarity search, chunking, context assembly, RAG prompts, evaluation, failure modes, advanced techniques, security, deployment, and performance.

    • 10.0 — Why RAG exists
    • 10.1 — What RAG is
    • 10.2 — Why RAG works
    • 10.3 — Architecture overview
    • 10.4 — Embeddings
    • 10.5 — Vector databases
    • 10.6 — Similarity search
    • 10.7 — Chunking
    • 10.8 — Context assembly
    • 10.9 — RAG prompt engineering
    • 10.10 — Evaluation
    • 10.11 — Failure modes
    • 10.12 — Advanced RAG techniques
    • 10.13 — Enterprise RAG
    • 10.14 — RAG security
    • 10.15 — Deployment
    • 10.16 — Performance optimization

12. Chapter 11: AI Safety, Alignment & Governance

    Bias, hallucination, prompt injection, jailbreaks, red-teaming, guardrails, and the safety lifecycle.

    • Core risks
    • Mitigation stack
    • Governance

13. Chapter 12: Capstone Project: Building a Complete LLM System

    The final demonstration of mastery. Students build a full-stack AI product: dataset, fine-tuning, RAG, inference API, chatbot UI, safety, monitoring, model card, technical report, and demo video.

    • 12.0 — Why the capstone matters
    • 12.1 — Capstone overview
    • 12.2 — Deliverables
    • 12.3 — Capstone architecture
    • 12.4 — Step 1: Build the dataset
    • 12.5 — Step 2: Fine-tune the model
    • 12.6 — Step 3: Build the vector database
    • 12.7 — Step 4: Build the RAG pipeline
    • 12.8 — Step 5: Build the inference API
    • 12.9 — Step 6: Build the chatbot interface
    • 12.10 — Step 7: Implement safety filters
    • 12.11 — Step 8: Monitoring & analytics
    • 12.12 — Step 9: Model card
    • 12.13 — Step 10: Technical report
    • 12.14 — Step 11: Demo video
    • 12.15 — Evaluation rubric

14. Chapter 13: Appendix A — Instructor Guide & Teaching Manual

    For instructors, TAs, partners, and licensees running cohort-based teaching, corporate training, or institutional workshops.

    • Purpose
    • Learning objectives by module
    • Lab instructions
    • Assessment rubrics
    • Suggested pacing
    • Instructor scripts
    • Troubleshooting guide

15. Chapter 14: Appendix B — Student Workbook & Lab Book

    Hands-on exercises for every module: setup, steps, expected outputs, reflection questions, and challenge tasks.

    • Module-by-module labs
    • Example: Lab 5.2 — Fine-Tune a Model with QLoRA
    • Workbook sections

16. Chapter 15: Appendix C — Portfolio Template

    A complete GitHub repo + portfolio website template so students ship like professional AI engineers.

    • GitHub repo structure
    • Portfolio website template
    • README template

17. Chapter 16: Appendix D — Mastery Checklist

    If you can do everything on this list, you are an LLM Engineer.

    • Foundations
    • Data engineering
    • Training
    • Fine-tuning
    • RAG
    • Deployment
    • Chatbot
    • Safety
    • Capstone

18. Chapter 17: Prompt & Context Engineering

    How to elicit the best behavior from an LLM through prompt design, structured outputs, and context packing — the highest-leverage skill in applied AI.

    • Why prompt engineering still matters
    • Core techniques
    • Structured outputs
    • Context engineering
    • Prompt injection & defenses
    • Evaluation

19. Chapter 18: Agents, Tools & MCP

    Going beyond single-turn chat: function calling, ReAct loops, multi-agent systems, the Model Context Protocol, and how to ship autonomous workflows safely.

    • What an agent actually is
    • Function calling & tool use
    • Agent architectures
    • Model Context Protocol (MCP)
    • Memory & state
    • Safety & control

20. Chapter 19: RLHF, DPO & Modern Alignment

    How frontier models are aligned with human preferences: reward modeling, PPO, DPO, GRPO, Constitutional AI, and RLAIF.

    • Why SFT is not enough
    • RLHF pipeline
    • Direct Preference Optimization (DPO)
    • GRPO & online RL
    • Constitutional AI & RLAIF
    • Evaluating alignment

21. Chapter 20: Multimodal Models

    Vision, audio, and video: how models extend beyond text and how to build multimodal applications.

    • Vision-language models
    • Audio & speech
    • Image & video generation
    • Multimodal RAG
    • Building multimodal apps

22. Chapter 21: Reasoning Models & Test-Time Compute

    The 2025 frontier: o1, o3, DeepSeek-R1, and the shift from scaling training to scaling inference.

    • What a reasoning model is
    • Test-time compute scaling
    • How reasoning models are trained
    • Verifiers & process reward models
    • When to use reasoning models

23. Chapter 22: Cost Economics & FinOps for LLMs

    Token economics, caching, model routing, and how to keep an LLM product profitable at scale.

    • Unit economics
    • Prompt caching
    • Model routing
    • Batch APIs & async
    • Prompt & context compression
    • Self-hosting vs. API

24. Chapter 23: Career & Interview Preparation

    How to land an AI engineering role: portfolio, system design interviews, common pitfalls, and the four major career tracks.

    • The four tracks
    • Portfolio that gets interviews
    • LLM system design interview
    • Common interview topics
    • Resume signals that work
    • Negotiation & comp