1.1 What Is an AI Agent — A Working Definition
An AI agent is a software system that pursues a goal autonomously by repeatedly perceiving its environment, reasoning about what to do, taking actions through tools, and updating its understanding from the results. The phrase that matters is 'pursues a goal' — agents are goal-directed, not turn-directed. A chatbot exists to answer the user's next message. An agent exists to close the gap between a current state and a desired state, and it will take as many model calls and tool calls as needed to do so, within the budgets you give it.
This definition has four moving parts, and each one is load-bearing. (1) Autonomously — the agent decides its own next step without per-step human approval. (2) Perceiving — the agent reads from the world: an inbox, a database, a sensor, a previous tool result. (3) Reasoning — the agent uses an LLM (or another policy) to choose what to do next. (4) Acting through tools — the agent emits structured calls into the world: HTTP requests, function calls, SQL, shell commands, code execution. Strip any one of these out and what remains is something simpler: a pipeline, a chatbot, a script, a workflow.
1.2 Why the Industry Suddenly Cares
Three forces converged in 2023–2025 to make agents practical. First, frontier LLMs crossed a reliability threshold for structured tool use: function calling, JSON mode, and constrained decoding mean a model can emit a tool invocation that actually parses and executes the first time, most of the time. Second, context windows grew from 4k–8k tokens to 128k–2M, enough to hold a meaningful working memory in-prompt. Third, the surrounding ecosystem matured: vector databases, browser-control APIs, sandboxed code execution, and standardized protocols (OpenAI tools, Anthropic tool use, MCP) made it possible to wire an LLM into the real world without a custom integration per tool.
The consequence is that work that used to require a human in the loop — triaging support tickets, reconciling invoices, investigating an alert, drafting and shipping a PR — can now be delegated to an agent, end to end, with the human reviewing outcomes rather than steps.
1.3 Agent vs Chatbot vs Workflow vs Copilot
These four words get used interchangeably and they shouldn't be. A chatbot is reactive and turn-based: user speaks, model answers, conversation ends. A workflow is a fixed graph of steps written by a developer: 'fetch → classify → route → respond' in a hard-coded order. A copilot suggests and asks; the human still drives. An agent is the only one of the four that decides its own next step at runtime based on what it just observed.
The practical test: if you can draw the full control flow as a flowchart before the system runs, it's a workflow, not an agent. If the control flow only becomes visible after the run by tracing the model's decisions, it's an agent. Most production 'agents' in the wild are actually workflows with an LLM somewhere inside, which is fine — but be honest about what you've built, because the failure modes are different.
1.4 The Five Anatomical Components
Every production agent, regardless of framework, exposes the same five components. (1) System prompt — the durable instruction that defines role, constraints, tools available, and termination conditions. (2) Memory — short-term working memory (the chat/scratchpad), long-term episodic memory (past runs), and semantic memory (vector store of domain knowledge). (3) Tools — the typed interface to the world: each tool has a name, a JSON schema for arguments, and a deterministic side effect. (4) Reasoning loop — the controller that calls the LLM, parses its output, dispatches tools, and decides whether to continue or stop. (5) Execution environment — the runtime that owns budgets, timeouts, sandboxing, logging, and tracing.
If any of these five is missing, your agent will fail in a predictable way. No system prompt → drift. No memory → amnesia between steps. No tools → it can talk but not act. No reasoning loop → it can act once but not adapt. No execution environment → it works in dev and explodes in prod.
1.5 The Sense → Think → Act → Observe Loop (Formal)
The agent loop is older than LLMs — it comes from the BDI (Belief-Desire-Intention) tradition in classical AI and from control theory. In its modern form: (1) Sense — gather the current observation: the user's new message, the last tool result, an external event. (2) Think — invoke the policy (the LLM) with the system prompt + memory + observation to produce a thought and an action. (3) Act — dispatch the chosen tool with the chosen arguments. (4) Observe — capture the tool's result, errors, and side effects, and append it to memory. Then return to step 1.
The loop terminates when one of three things happens: the model emits a 'finish' action with a final answer; a budget (time, tokens, dollars, steps) is exhausted; or a guardrail trips (policy violation, dangerous action, low-confidence threshold). A well-engineered loop logs every iteration with enough detail that you can replay the entire run from the trace alone.
1.6 ReAct: The Pattern That Made Agents Work
ReAct (Yao et al., 2022) is the prompting pattern that took agents from research demo to production. The model is asked to interleave Thought, Action, and Observation lines: 'Thought: I need yesterday's revenue. Action: query_db({sql: ...}). Observation: 12,431.' The thought line is critical — it forces the model to commit to a reason before it acts, which dramatically reduces tool-call hallucination and makes traces auditable.
In modern frameworks (LangGraph, OpenAI Assistants, Anthropic tool use), the thought is often hidden in the model's reasoning tokens rather than printed, but the structure is the same. When you debug a misbehaving agent, the first thing you read is the thought log: if the thoughts are reasonable and the actions don't match them, your tool schemas are wrong; if the thoughts are nonsense, your system prompt or context is wrong.
1.7 Goals, Subgoals, and the Planning Hierarchy
A goal is a measurable target state — 'the user's invoice is sent, marked paid in Stripe, and logged in the GL'. Real goals decompose into subgoals: fetch invoice, charge card, write ledger entry, email receipt. Agents handle this in one of three ways. (1) Implicit decomposition — the model figures out subgoals on the fly each loop iteration (fast, fragile). (2) Explicit planner — a separate LLM call produces a written plan first, then the executor follows it (slower, more controllable). (3) Hierarchical agents — a manager agent owns the goal and dispatches worker agents per subgoal (powerful, expensive, hard to debug).
For your first ten production agents, use the explicit planner pattern. The plan becomes an artifact you can show to the user for approval before any side effects happen, which is half the safety story.
1.8 The Tool Boundary Is the Most Important Boundary
Everything an agent does in the real world flows through tools, so the design of your tool surface determines almost everything about how your agent behaves. Three rules to internalize on day one. (1) Tools should be small and orthogonal — one tool per capability, not one mega-tool that takes a 'mode' parameter. (2) Tool schemas should be self-documenting — names, descriptions, and field descriptions are part of the prompt the model reads; vague names produce vague calls. (3) Tools should fail loudly and structurally — a tool that returns 'error: something went wrong' teaches the model nothing; a tool that returns `{error: 'not_found', resource: 'invoice_42'}` lets the model recover.
The most common production bug in agent systems is not a model failure, it is a tool-schema failure: an optional field the model keeps omitting, an enum the model keeps misspelling, a result shape the model can't parse. Spend more time on tool design than on prompt engineering.
1.9 Memory: The Difference Between a Demo and a System
A demo agent uses only the in-context chat history. A production agent has at least three memory tiers. (1) Working memory — the current loop's scratchpad, ephemeral, lives in the prompt. (2) Episodic memory — past runs, indexed by user/session/task, used to recall 'last time we ran this, the API returned X'. (3) Semantic memory — durable domain knowledge (your docs, your schemas, your policies) usually backed by a vector store and surfaced by retrieval.
The failure mode of skipping memory tiers is unmistakable: the agent re-asks questions you already answered, re-runs queries that already failed, re-discovers the same policy violation. Memory is what turns an agent from a goldfish into a colleague, and it is covered in depth in Module 3.
1.10 The Agent Lifecycle, Phase by Phase
A single agent run moves through six phases and you should be able to name all six in your sleep. (1) Initialization — load system prompt, hydrate memory, validate tool availability. (2) Goal interpretation — parse the user request into an internal goal representation. (3) Planning — produce an explicit or implicit plan. (4) Execution — run the sense/think/act loop against the plan. (5) Evaluation — score the outcome against the goal (did we succeed? partially? not at all?). (6) Termination — emit the final response, persist the trace, release resources.
Production systems wrap this lifecycle in a job runner with retries, idempotency keys, dead-letter queues, and human-handoff hooks. Treating the lifecycle as a first-class object — not just 'the function we called' — is what makes agent systems operable.
1.11 Budgets, Guardrails, and Termination
Unbounded loops are the original sin of agent engineering. Every production agent must declare, at construction time, four budgets: max steps (e.g. 25), max wall-clock time (e.g. 5 min), max tokens (e.g. 200k), max dollars (e.g. $1). Each loop iteration decrements all four, and any one hitting zero terminates the run with a clear reason code.
Guardrails are the second termination class: policy violations (the model tried to call a forbidden tool), confidence floors (the model said 'I'm not sure' three times in a row), and human-in-the-loop triggers (the proposed action exceeds a risk threshold). A well-engineered agent terminates loudly with a structured reason, never silently or by running out of tokens mid-thought.
1.12 When NOT to Build an Agent
Agents are expensive, slow, and non-deterministic. If your task has a fixed sequence of steps, write a workflow. If your task is a single transformation, write a function. If your task is a question-and-answer, write a RAG chatbot. The honest test: enumerate the runtime decisions your task requires. If the count is zero or one, you do not need an agent. If the count is variable and depends on intermediate results you can't see ahead of time, you do.
The second test is value-per-decision. Agents shine when each runtime decision saves significant human time — a triage decision, an investigation step, a refactor choice. Agents waste money when the decisions are trivial and the LLM markup over a hand-written switch statement is pure cost.
1.13 Failure Modes You Will See in Your First Week
Six failure modes account for ~80% of early agent bugs. (1) Tool-call loops — the agent calls the same tool with the same arguments forever; fix with progress tracking and step budgets. (2) Argument hallucination — the agent invents fields the tool doesn't accept; fix with strict JSON schemas and a re-prompt on parse failure. (3) Context overflow — memory grows until the prompt exceeds the window; fix with summarization and sliding windows (Module 3). (4) Premature termination — the agent says 'done' before the goal is met; fix with explicit success criteria in the system prompt. (5) Silent drift — the agent slowly forgets the goal across long runs; fix with periodic goal re-injection. (6) Sycophancy — the agent agrees with bad user instructions; fix with constitution-style constraints (Module 7).
Log every loop iteration with its thought, action, observation, and budget state. Half of debugging an agent is being able to scroll back through a clean trace.
1.14 What You Will Build in This Diploma
By the end of Module 12 you will have built, evaluated, deployed, and operated a production-grade agent of your own choosing. Along the way you will have written: a ReAct loop from scratch (Module 2), a three-tier memory system (Module 3), a tool registry with strict schemas (Module 6), an eval harness with golden traces (Module 8), a deployment pipeline with observability (Modules 9–10), and a capstone artifact you can publish to the GACS portfolio.
This module's job was to give you the vocabulary: agent, loop, tool, memory, lifecycle, budget, guardrail. Internalize these twelve words. Every later module sits on top of them.
