LLM Engineering Fundamentals — Free Intro

A large language model is a function that takes a sequence of tokens and predicts the probability distribution of the next token. That's it. Everything else — chat, code generation, agentic behaviour, reasoning — is built on top of that one operation, applied repeatedly. The model itself is a stack of transformer blocks, typically tens to hundreds of them, each containing attention layers and feed-forward layers, with billions of learned parameters connecting them. The intelligence isn't in any single component; it emerges from training that whole stack on enormous quantities of text until predicting the next token becomes startlingly good at modelling how humans think on paper.

The model doesn't see English. It sees integers. A tokenizer splits raw text into pieces — sometimes whole words, often sub-word fragments, sometimes individual characters — and maps each to a number from a fixed vocabulary, typically 30k–200k entries. The phrase "unbelievable" might become three tokens: "un", "believ", "able". This sounds like an implementation detail. It isn't. Token boundaries affect cost (you pay per token), latency (more tokens = more compute), and quality (rare words become many tokens, common ones become few). It also affects what languages and code the model handles well — a tokenizer trained mostly on English will burn far more tokens to represent the same idea in Thai or Korean.

Each token ID gets mapped to a vector — typically hundreds or thousands of numbers long. These vectors live in a high-dimensional space where similar concepts cluster together. The classic example: the vector for "king" minus the vector for "man" plus the vector for "woman" lands close to the vector for "queen". The model doesn't reason about words; it does arithmetic on these vectors.

The embedding layer is the first thing in the model and one of the most important. It's also the basis of every retrieval-augmented system you've heard of: "semantic search" is literally storing document embeddings and finding the ones closest to your query embedding.

Before transformers, models read text strictly left-to-right and struggled to connect words far apart. The 2017 transformer paper introduced attention: for every token, look at every other token in the context and decide how relevant each one is to predicting the next output. This sounds expensive — and it is, costing roughly O(n²) where n is the context length — but it lets the model handle long-range dependencies natively.

Multi-head attention runs this lookup in parallel many times, each head learning to attend to different kinds of relationships (syntactic, semantic, positional). Stack dozens of these layers and you get the modern LLM.

Three distinct training stages produce a modern LLM:

1. Pretraining: feed it a trillion+ tokens of raw web text, books, and code. It learns to predict the next token. Output: a model that can complete text but doesn't follow instructions well.

2. Supervised fine-tuning (SFT): show it tens of thousands of (instruction, ideal response) pairs. It learns the "reply helpfully" pattern.

3. RLHF / preference optimisation: ask humans (or another model) to rank pairs of responses, and train the model to produce the preferred kind. This is where personality and tone come from.

For most product work, you don't pretrain — too expensive. You fine-tune existing open-weight models with LoRA or QLoRA on a few thousand examples, which is achievable on a single consumer GPU.

Three tools, often confused, each suited to a different problem.

Prompting (just give better instructions) is fast and free. Use it when behaviour is correct but inconsistent, or when you can describe the rule in plain language.

Retrieval-Augmented Generation (RAG) is when the answer depends on data the model wasn't trained on — your private docs, your product manual, last week's customer tickets. You embed those documents, search the relevant ones at query time, and feed them into the prompt. Use this when the bottleneck is knowledge, not behaviour.

Fine-tuning is when you need the model to behave differently — a specific format, a specific tone, a specific kind of reasoning — consistently and at low cost per call. It's not the right tool for adding facts; it's the right tool for changing style or specialising a skill.

The hardest part of LLM engineering is not building the thing, it's knowing whether the thing is good. Traditional accuracy metrics fail because there is rarely a single correct answer. The modern toolkit:

- Task-specific benchmarks (e.g. HumanEval for code, MMLU for general knowledge). - Held-out evaluation sets you write yourself, scored either by humans or by a stronger model acting as judge. - A/B tests in production with real users. - Regression suites that catch when a prompt change breaks something that used to work.

Without evals you are flying blind. Every team that ships LLM features in production reaches the same conclusion: eval infrastructure is more valuable than the latest model.

A trained model is useless until it serves requests fast and cheap. The main levers:

Quantization: store weights in 8-bit or 4-bit instead of 16/32-bit. Cuts memory roughly 2–4×, with small quality loss on most tasks. Almost always worth it.

Batching: serve multiple requests through the same forward pass. Drastically improves throughput at the cost of slightly higher latency per request.

KV-caching: don't recompute attention for tokens you already processed. Standard in every modern serving framework.

Model-size choice: the cheapest 30% accuracy improvement is often switching to a smaller model for the 80% of requests that don't need a frontier model. Route hard requests to the big model, easy ones to the small one.

The cost spreadsheet beats the architecture lecture. Always.