How To Build AI Agents That Actually Ship

Most people build AI agents that work in demos and break in production.

This is the guide for the other kind.

Not the kind you screenshot and post. The kind that runs at 2 AM without you watching it, handles edge cases you forgot existed, and still returns the right answer on attempt 4,000.

I'm going to show you exactly how to build one. From architecture to error handling to evaluation. Real code. Real decisions. No theory without application.

Save this. You'll come back to it.

What an agent actually is

Before you build anything, kill the mythology.

An agent is not magic. It is not a personality. It is not sentient.

An agent is a loop.

while goal_not_complete:
    observation = get_current_state()
    action = model.decide(observation, tools, memory)
    result = execute(action)
    update_state(result)

That's it. A model with tools, running in a loop, until it finishes or fails.

Everything else, the frameworks, the abstractions, the branded names, is built on top of that loop.

Understand the loop and you understand every agent you'll ever build or debug.

PART 1 - Architecture First

Most tutorials skip this and go straight to code. That's why most tutorials produce agents that break.

Architecture is the decision you make before you write a line. It determines everything that follows.

Single-agent vs multi-agent

A single agent handles one goal with multiple tools. Simpler to build, simpler to debug, right for most problems.

A multi-agent system has specialized agents that hand off work to each other. More powerful, dramatically harder to make reliable. Do not start here.

Build single-agent first. Graduate to multi-agent when you have a specific problem that genuinely needs it, not because it sounds more impressive.

The 3 things every agent needs

1. A goal it can evaluate
2. Tools it can use to make progress
3. A way to know when it's done

If any of these is vague, your agent will spin, hallucinate completion, or run until you cut it off.

Before writing code, write these 3 things in plain English. If you can't, your agent isn't ready to build yet.

PART 2 - Build the Foundation

Let's build a research agent. It takes a question, searches for information, reads sources, synthesizes an answer, and cites what it found.

Real problem. Real tools. Real failure modes.

Step 1: Define your tools

Tools are functions the model can call. Every tool needs a name, a description the model reads, input parameters, and a return value.

import anthropic
import requests
from typing import Any

client = anthropic.Anthropic()

tools = [
    {
        "name": "web_search",
        "description": "Search the web for current information on a topic. Returns a list of results with titles, URLs, and snippets.",
        "input_schema": {
            "type": "object",
            "properties": {
                "query": {
                    "type": "string",
                    "description": "The search query"
                },
                "num_results": {
                    "type": "integer",
                    "description": "Number of results to return (1-10)",
                    "default": 5
                }
            },
            "required": ["query"]
        }
    },
    {
        "name": "read_url",
        "description": "Read the full text content of a webpage. Use this to get detailed information from a specific URL.",
        "input_schema": {
            "type": "object",
            "properties": {
                "url": {
                    "type": "string",
                    "description": "The URL to read"
                }
            },
            "required": ["url"]
        }
    },
    {
        "name": "save_finding",
        "description": "Save an important finding to memory for use in the final answer.",
        "input_schema": {
            "type": "object",
            "properties": {
                "finding": {
                    "type": "string",
                    "description": "The finding to save"
                },
                "source": {
                    "type": "string",
                    "description": "The URL or source of this finding"
                }
            },
            "required": ["finding", "source"]
        }
    }
]

Notice what makes these tools good:

• Descriptions tell the model when to use them, not just what they do
• Parameters have clear types and descriptions
• Return values will be predictable and parseable

Bad tool descriptions produce bad tool calls. This is where most agents break first.

Step 2: Build the execution layer

def execute_tool(tool_name: str, tool_input: dict, memory: list) -> Any:
    
    if tool_name == "web_search":
        # Replace with your actual search API
        results = mock_search(tool_input["query"], tool_input.get("num_results", 5))
        return results
    
    elif tool_name == "read_url":
        try:
            response = requests.get(tool_input["url"], timeout=10)
            # In production: parse HTML, extract text, truncate to token budget
            return {"content": response.text[:3000], "status": response.status_code}
        except requests.Timeout:
            return {"error": "Request timed out", "url": tool_input["url"]}
        except Exception as e:
            return {"error": str(e), "url": tool_input["url"]}
    
    elif tool_name == "save_finding":
        memory.append({
            "finding": tool_input["finding"],
            "source": tool_input["source"]
        })
        return {"saved": True, "total_findings": len(memory)}
    
    else:
        return {"error": f"Unknown tool: {tool_name}"}

Error returns, not exceptions. Every tool should return something the model can read and recover from. An exception crashes the loop. A descriptive error lets the model try again.

PART 3 - The Agent Loop

def run_agent(question: str, max_iterations: int = 15) -> dict:
    
    memory = []
    messages = [
        {
            "role": "user",
            "content": f"""Research this question thoroughly and provide a well-sourced answer.

Question: {question}

Instructions:
- Search for relevant information
- Read the most promising sources in full
- Save your key findings as you go
- After gathering enough information, provide a comprehensive answer
- Cite your sources"""
        }
    ]
    
    system_prompt = """You are a research agent. Your job is to find accurate, sourced answers to questions.

Work systematically:
1. Start with broad searches to understand the landscape
2. Narrow to the most relevant sources
3. Read full content when a snippet isn't enough
4. Save findings before you forget them
5. Stop searching when you have enough to answer confidently

When you have sufficient information, provide your final answer directly without using more tools."""
    
    iteration = 0
    
    while iteration < max_iterations:
        iteration += 1
        print(f"Iteration {iteration}")
        
        response = client.messages.create(
            model="claude-opus-4-6",
            max_tokens=4096,
            system=system_prompt,
            tools=tools,
            messages=messages
        )
        
        # Add assistant response to conversation history
        messages.append({
            "role": "assistant",
            "content": response.content
        })
        
        # Check if agent is done
        if response.stop_reason == "end_turn":
            final_text = next(
                (block.text for block in response.content if hasattr(block, "text")),
                "No response generated"
            )
            return {
                "answer": final_text,
                "sources": memory,
                "iterations": iteration
            }
        
        # Process tool calls
        if response.stop_reason == "tool_use":
            tool_results = []
            
            for block in response.content:
                if block.type == "tool_use":
                    print(f"  → Calling {block.name}")
                    result = execute_tool(block.name, block.input, memory)
                    
                    tool_results.append({
                        "type": "tool_result",
                        "tool_use_id": block.id,
                        "content": str(result)
                    })
            
            messages.append({
                "role": "user",
                "content": tool_results
            })
    
    # Hit max iterations
    return {
        "answer": "Research incomplete: reached maximum iterations",
        "sources": memory,
        "iterations": iteration,
        "error": "max_iterations_reached"
    }

3 things this loop gets right that most tutorials skip:

Conversation history is maintained. The model sees everything that happened before. Without this, every iteration is stateless and the agent can't build on previous steps.

Tool results go back as user messages. This is the correct API pattern. Tool results are not assistant messages.

Max iterations is a hard limit, not a suggestion. An agent without a ceiling will run forever if it gets confused. $40 API bills have happened. Set the limit.

PART 4 - Failure Handling

This is where agents actually die. And it's almost never covered in tutorials.

The 4 failure modes you will hit

1. The hallucinated completion. The model says it's done before it actually is. Fix: include a completion checklist in your system prompt.
2. The infinite search loop. The model keeps searching instead of synthesizing. Fix: track search count and prompt the model to move toward an answer after N searches.
3. The tool error spiral. One tool fails, the model tries again, fails again, loops. Fix: return descriptive errors and instruct the model to move on after 2 failures.
4. The context explosion. Long tool results fill the context window and the model degrades. Fix: truncate tool outputs, summarize long content before returning it.

def execute_tool_with_limits(
    tool_name: str, 
    tool_input: dict, 
    memory: list,
    attempt_tracker: dict
) -> Any:
    
    # Track attempts per tool+input combination
    tool_key = f"{tool_name}:{str(tool_input)}"
    attempt_tracker[tool_key] = attempt_tracker.get(tool_key, 0) + 1
    
    if attempt_tracker[tool_key] > 2:
        return {
            "error": f"This exact call has failed {attempt_tracker[tool_key]} times. Try a different approach or different parameters.",
            "tool": tool_name
        }
    
    result = execute_tool(tool_name, tool_input, memory)
    
    # Truncate large outputs
    result_str = str(result)
    if len(result_str) > 8000:
        result_str = result_str[:8000] + "\n\n[Content truncated. Use save_finding to preserve key information before it's lost.]"
        return result_str
    
    return result

PART 5 - Evaluation

An agent you can't measure is an agent you can't improve.

Most builders skip evaluation entirely. They ship, something breaks in production, they don't know what or why, and they spend a week firefighting.

Build the eval before you scale.

import json

def evaluate_research_agent(test_cases: list) -> dict:
    
    results = []
    
    for case in test_cases:
        print(f"Testing: {case['question'][:60]}...")
        
        output = run_agent(case["question"])
        
        # Score the result
        score = {
            "question": case["question"],
            "completed": "error" not in output,
            "has_sources": len(output.get("sources", [])) > 0,
            "iterations_used": output.get("iterations", 0),
            "answer_length": len(output.get("answer", "")),
        }
        
        # Check if expected keywords appear in answer
        if "expected_keywords" in case:
            keywords_found = sum(
                1 for kw in case["expected_keywords"] 
                if kw.lower() in output.get("answer", "").lower()
            )
            score["keyword_coverage"] = keywords_found / len(case["expected_keywords"])
        
        results.append(score)
    
    summary = {
        "total_cases": len(results),
        "completion_rate": sum(r["completion_rate"] for r in results) / len(results),
        "avg_iterations": sum(r["iterations_used"] for r in results) / len(results),
        "source_rate": sum(1 for r in results if r["has_sources"]) / len(results)
    }
    
    return {"results": results, "summary": summary}

# Example test cases
test_cases = [
    {
        "question": "What is the current state of AI regulation in the EU?",
        "expected_keywords": ["AI Act", "regulation", "2024"]
    },
    {
        "question": "How does transformer attention work?",
        "expected_keywords": ["attention", "query", "key", "value"]
    }
]

Run this on every meaningful change. Compare scores. If a change improves latency but drops keyword coverage, that's a tradeoff worth knowing about, not discovering in production.

PART 6 - Moving to Multi-Agent

You've built a single agent that works. Now, and only now, consider multi-agent.

The pattern that works: an orchestrator that plans and delegates, plus specialized workers that execute.

def orchestrator(complex_task: str) -> dict:
    
    # Orchestrator breaks the task into subtasks
    plan_response = client.messages.create(
        model="claude-opus-4-6",
        max_tokens=1024,
        system="You are a task planner. Break complex tasks into clear subtasks that specialized agents can execute. Return JSON.",
        messages=[{
            "role": "user",
            "content": f"Break this into subtasks: {complex_task}\n\nReturn: {{\"subtasks\": [{{\"id\": 1, \"task\": \"...\", \"type\": \"research|write|analyze\"}}]}}"
        }]
    )
    
    plan = json.loads(plan_response.content[0].text)
    
    results = {}
    for subtask in plan["subtasks"]:
        if subtask["type"] == "research":
            results[subtask["id"]] = run_agent(subtask["task"])
        # Add other specialized agents here
    
    # Orchestrator synthesizes
    final = client.messages.create(
        model="claude-opus-4-6",
        max_tokens=4096,
        messages=[{
            "role": "user",
            "content": f"Original task: {complex_task}\n\nSubtask results: {json.dumps(results)}\n\nSynthesize into a final comprehensive output."
        }]
    )
    
    return {"output": final.content[0].text, "subtask_results": results}

The orchestrator pattern is powerful because each worker can be specialized, evaluated, and improved independently. When the research agent gets better, the whole system gets better.

What you actually need to ship

A portfolio of shipped agents beats a year of tutorials.

Build these 3, in order:

Project 1. A single-agent research tool with evaluation. The one from this guide. Get it deployed somewhere a real person can use it. Measure completion rate and source quality.

Project 2. A document Q&A agent with RAG. Takes a folder of PDFs, answers questions about them, cites page numbers. Add eval: does it find the right chunk? Does the answer match the source?

Project 3. A multi-agent system that solves a real workflow. Pick something you do manually and automate it. An orchestrator and 2 workers minimum. Document what broke and how you fixed it.

3 deployed projects with documented evaluations is a portfolio that opens doors most CS graduates can't open.

The part nobody tells you

The hardest part of building agents is not the code.

It's knowing when your agent is wrong.

An agent that fails loudly is easy to fix. An agent that confidently returns a plausible but incorrect answer is the one that costs you users, reputation, or money.

Build your evals before you scale. Measure what matters. Trust the numbers, not the demo.

The teams that ship reliable agents aren't the ones who found a better framework.

They're the ones who decided that "it works in the demo" was never good enough.

Start there, and everything else follows.