AI Agents
Systems that decide and act — and fail gracefully when they decide wrong.
An agent is a loop: the model picks a tool, something happens in the world, the result goes back into context, repeat until done. That is the whole idea — which is why agents are trivial to prototype and brutal to operate. Every iteration compounds error, spends money, and takes an action somewhere that may not be reversible. This path is built around that reality. You begin by writing the loop yourself, without a framework, so you can see exactly where control lives and what the model is actually deciding. Then tools, which are an API design problem in disguise: a tool with a vague name and a fuzzy description will be called at the wrong moment, and no amount of prompting repairs a bad interface. Then the parts nobody demos — memory that does not turn into a landfill, planning that survives a failed step, and the discipline of assuming the agent will eventually do something stupid, because it will. An entire phase goes to failure: retries, loops, budget caps, approval gates. In production an agent’s ceiling is not how clever it is, it is how gracefully it breaks. Multi-agent architectures come last, with a warning: most teams reach for them to avoid fixing a single agent they never instrumented.
What you will learn
Prerequisites
- Solid Python or TypeScript, including async
- You have already built something against an LLM API
- Understanding of RAG and structured output
- Patience for debugging non-deterministic systems
Where it leads
- AI Agent Engineer
- Automation Architect
- Applied AI Engineer
- Technical lead on agent products
Phases
Phase 1 — The loop and its tools
Build the agentic loop by hand, then design tools a model can actually use correctly.
Estimated duration · 8-10 weeksWriting the loop yourself
The loop is about eighty lines: send context, read the tool call, execute it, append the result, repeat. Writing it yourself is not an exercise in purity — it is the only way to know where your bug lives once the agent starts behaving strangely on iteration seven. You will also confront stop conditions, which are harder than they look: an agent that does not know it is finished will happily spend your budget proving it.
Topics covered
What you will build
- Write a working agent loop in under 100 lines with no agent framework — three tools, a hard iteration cap, and a transcript you can read
- Deliberately induce an infinite loop, then fix it with a stop condition rather than a lower cap
- Print the exact context sent on every iteration and find the point where it stopped making sense
Tool design is API design
The model is a junior developer reading your documentation and nothing else. It cannot ask a colleague, it will not read your source, and it takes your description literally. Ambiguous names, overlapping tools and error messages that just say "failed" produce agents that thrash. Rewriting a tool description usually beats rewriting the system prompt, and it is a fraction of the work.
Topics covered
What you will build
- Measure wrong-tool-call rate over 50 tasks, rewrite only the descriptions, and report the delta
- Rewrite every tool error into an actionable message and show the agent now recovers from a failure it previously looped on
- Split one overloaded tool into three focused ones and prove the success rate moved
Phase 2 — Memory and planning
Give the agent state and strategy without drowning it in its own history.
Estimated duration · 8-10 weeksMemory that does not become a landfill
The naive design appends everything forever, and it fails twice: cost grows linearly while quality falls, because a context stuffed with twenty stale tool outputs buries the three that matter. Memory is a budgeting problem. The skill is deciding what to forget on purpose — and most teams never write that rule down, which is why their agent degrades over a long session and nobody can say why.
Topics covered
What you will build
- Run one agent for 50 iterations and chart cost and quality per step until you can see the degradation point
- Add compaction and hold quality flat while cutting context growth to a fraction of linear
- Write an explicit eviction policy for your agent, then find the task it breaks and defend or revise the rule
Planning, decomposition and recovery
Plan-then-act looks disciplined and shatters on the first surprise. Pure reactivity wanders. The useful answer is a deterministic scaffold around non-deterministic steps: you decide the shape of the work in code, the model decides the content. Replanning after a failed step is where agents earn their keep — and self-critique helps less than people claim, because a model that made an error is not well placed to notice it.
Topics covered
What you will build
- Take one agent task and rebuild it as a fixed workflow — measure both, then argue in writing which one should ship
- Inject a failing tool at step three and make the agent replan instead of retrying identically
- Add a verification step after every write action and count how many bad actions it catches over 30 runs
Phase 3 — Failure, guardrails, multi-agent
Assume the agent will do something stupid. Make that survivable, observable and cheap.
Estimated duration · 8-12 weeksGuardrails and the blast radius
Security for agents is not a prompt that says "be careful". It is least privilege, sandboxes and approval gates on anything you cannot undo. The attack surface most teams miss is the tool result: a document your agent retrieves can contain instructions, and your agent reads them with the same trust as yours. Treat every tool output as hostile input, because eventually one will be.
Topics covered
What you will build
- Plant an injection payload inside a document your agent retrieves, watch it obey, then design the mitigation and prove it holds
- Put an approval gate on every irreversible action and measure how often a human says no — if it is zero, your gate is theatre
- Build a working kill switch and stop a live agent mid-tool-call without corrupting its state
Observability, evaluation and multi-agent
You cannot debug what you cannot replay. Trace every step, every tool call, every token, and make a past run reproducible — otherwise every incident is folklore. Evaluate trajectories, not just final answers: an agent that reached the right result through six wasted calls and one dangerous one is not working. Multi-agent belongs here, at the end, because it multiplies everything in this step.
Topics covered
What you will build
- Build a trace viewer that lets you replay any past run step by step and hand it to a colleague who then debugs a failure you did not explain
- Score 30 runs on trajectory quality, not just outcome, and find at least one "successful" run you would refuse to ship
- Split one agent into two, measure cost and success against the single-agent baseline, and write the honest verdict even if it is negative
Questions
Should I use an agent framework?
Eventually, yes — but not first. Write the loop by hand once, because a framework does not remove the difficulty, it relocates it: your agent still loops, still burns budget, still calls the wrong tool, and now the code doing it belongs to someone else. Engineers who started inside a framework can produce a demo on day one and then stall for weeks on a bug they cannot see, because the interesting decisions are behind an abstraction. Once you have written your own, adopt a framework for what it genuinely gives you — retries, tracing, concurrency, integrations — with the ability to read what it does and replace it when it stops fitting.
Why do agents work in demos and fall apart in production?
Because a demo is one happy path and production is a distribution. Errors compound: a step that is right 95% of the time is right about 60% of the time after ten steps, and the demo only ever showed you the run that worked. The missing pieces are always the same three — tool errors the model cannot act on, so it retries identically; no budget or iteration cap, so a confused agent spends real money in a circle; and evaluation on final answers only, which hides trajectories that got lucky. None of that is fixed by a better model. It is fixed by instrumenting the loop and being honest about the numbers.
When is a multi-agent architecture actually worth it?
Less often than the diagrams suggest. Two reasons hold up: genuine parallelism, where independent subtasks can run at the same time and you actually save wall-clock; and genuine separation, where sub-jobs need different tools or different permissions and you want the blast radius split. Everything else is usually a single agent that was never instrumented, dressed up as an org chart. Adding agents adds a coordination tax — handoffs lose context, debugging now spans several transcripts, and cost multiplies quietly. Fix the one agent first: better tool descriptions, a memory policy, trajectory evals. If it still fails, split it deliberately and measure against the single-agent baseline.
Related paths
Want to walk this path with a coach and a cohort?
Discover the 212AY Academy →