An interactive walkthrough of Barry Zhang's AI Engineer Summit talk. Three ideas anchor it: don't build agents for everything, keep it simple, and think like your agent — plus the spectrum that runs from simple features to workflows to fully autonomous agents, and the open questions coming next.
Most of us started with simple features — summarization, classification, extraction — single model calls that felt like magic a few years ago and are now table stakes. As products matured, one call wasn't enough, so we started orchestrating multiple model calls in predefined control flows. Barry calls these workflows, and considers them the beginning of agentic systems.
Now models are capable enough that agents can decide their own trajectory and operate almost independently from environment feedback. What the next phase looks like — more general single agents, or multi-agent collaboration — is still too early to name.
The distinction is precise. A workflow runs model calls through a predefined control flow — you drew the path, it always follows it. An agent runs the model in a loop, deciding its own next step based on what the environment gives back. Same building blocks; who's steering is different.
Agents scale complex and valuable tasks — they're not a drop-in upgrade for every use case. Barry's checklist has four questions. Toggle each one below and watch the recommendation change. If you can't check the first box, an explicit workflow is almost always the better, cheaper, more controllable choice.
To Anthropic, an agent is just a model using tools in a loop. Three components define it: the environment it operates in, the tools that let it act and get feedback, and the system prompt that sets goals, constraints, and ideal behavior. Then the model is called in a loop. That's it.
Wildly different agents — coding, search, computer use — look nothing alike on the surface but share almost the exact same backbone and code. The only real design decisions are which tools and what prompt. Any complexity up front kills iteration speed; optimizations (caching trajectories, parallelizing tool calls, surfacing progress for user trust) come after the behavior is right.
Builders design agents from their own perspective, then get confused when the agent errs. At each step the model is just running inference on a very limited context — everything it knows about the world lives in roughly 10–20k tokens. Limiting yourself to that context reveals whether it's actually sufficient and coherent.
Barry's exercise: be a computer-use agent. You get a static screenshot and a poorly written task. You reason, then attempt a click — but while inference and tool execution run, it's like closing your eyes for 3–5 seconds and using the computer in the dark. You open them to a new screenshot. Did it work? You just don't know. Try the blind cycle below, then give the agent the context it actually needed.
Barry closes with the open problems on his mind — the things AI engineers still need to figure out together. Click each to see the idea and the question it leaves open.
Agency rises along a spectrum (features → workflows → agents) and so do cost, latency, and risk → so don't build agents for everything; run the four-part checklist → when you do, keep it simple: a model using tools in a loop → and as you iterate, think like your agent by living in its context window. Now go build.
No spam — just new posts and learning pages when they ship.
Drop your email and I will send new posts and learning pages your way.
Thanks — you are subscribed.
You can close this dialog now.
We will only email you when there is something new. Unsubscribe any time.