Paul SerbanSoftware Engineer

Agent Harness Engineering, Loop Engineering, and Graph Engineering: The Architecture of Reliable AI AgentsWhy the scaffolding around a model, not the model itself, decides whether an agent survives production

Introduction

Ask a team that has shipped a production AI agent what actually took the engineering time, and very few will say "the model". Model selection matters, but it is usually a single configuration line. What consumes weeks of work is everything wrapped around that line: which tools the model is allowed to call and under what permissions, how many times it is allowed to retry a failing step before a human is paged, and how a dozen of these steps are wired together into something that behaves predictably instead of wandering. That wrapping has acquired its own vocabulary over the last two years, and it is worth taking seriously rather than treating as jargon, because each term names a genuinely distinct engineering responsibility.

This article works through three of those terms in the order they tend to matter as a system grows:

  • agent harness engineering: the design of the runtime environment - tools, permissions, context management - that turns a raw model into something that can act;
  • loop engineering: the control flow inside that runtime that decides when the agent keeps going, stops, or escalates;
  • graph engineering: the practice of composing multiple agents, tools, and checkpoints into an explicit, inspectable workflow once a single loop is no longer enough;

None of these is a framework you install. They are architectural concerns you are responsible for whether or not you name them, and naming them is what makes it possible to reason about which one is broken when a system misbehaves.

The Problem: Why an Agent Is More Than a Model Call

The earliest agent demos reinforced a misleading intuition: give a model a system prompt, a handful of tool definitions, and a while-loop, and it will figure out the rest. This works for a five-minute demo against a clean dataset. It stops working once the agent runs for forty-five minutes against a real codebase or a real customer database, because at that point the failure modes are no longer about model quality - they are about infrastructure. What happens when a tool call times out. What happens when the conversation history exceeds the context window halfway through a task. What happens when the model tries to delete a table it should never have write access to. None of these are prompting problems, and no amount of instruction tuning in the system prompt reliably solves them, because the model cannot enforce a permission boundary on itself from inside its own output.

This is the gap that agent harness engineering fills. Anthropic's engineering team has published two detailed accounts of this work - one on building effective harnesses for long-running agents, and one on the general design patterns behind harnessing a model's intelligence - and both describe the harness as the durable software layer between the model and the world: the tool surface, the permission model, the context management strategy, and the execution environment. The Claude Agent SDK is explicitly described by Anthropic as a general-purpose harness, separate from the model it wraps, precisely because the same underlying model produces very different agents depending on what harness surrounds it. A coding agent and a customer-support agent built on the same model differ almost entirely in their harness, not in their prompt.

Once a harness exists, a second problem appears inside it: deciding what happens across repeated calls to the model. A harness that hands the model a tool and lets it call that tool exactly once is not an agent, it is a function call with extra steps. Real agents iterate - they observe a tool's output, decide whether the task is done, and either stop or continue - and that iteration has to be governed by something other than the model's own judgment about when to stop, which is where loop engineering enters. And once an application needs more than one agent, or more than one distinct phase of work (research, then draft, then verify, then send), a single loop stops being an adequate representation of the system, which is where graph engineering becomes the relevant layer. Each of these three concerns builds on the one below it, and each becomes visible only once the layer beneath it is reasonably solid.

The Three Layers: A Deep Technical Explanation

These three layers are easiest to reason about as increasing scopes of responsibility. A harness governs a single agent's relationship with its environment. A loop governs the control flow of a single agent's execution over time. A graph governs how multiple agents, tools, and checkpoints relate to each other across an entire application. Confusing the layers is common and costly: teams debug a permissions failure by rewriting the loop, or debug a coordination failure between two agents by adding more tools to one harness, when the actual defect lives one layer up or down from where they are looking.

Agent Harness Engineering: The Runtime Around the Model

An agent harness is the software scaffolding that sits between a language model and everything it is allowed to affect: the file system, a database, an API, another service. It owns the tool registry (which functions the model can call and how they are described to it), the permission model (which of those calls require approval, which are auto-approved, which are forbidden outright), and the context management strategy (how conversation history, tool results, and any persisted state are kept within the model's usable context window over a long-running session). Anthropic's public write-up on long-running agent harnesses describes techniques such as context compaction, periodic full resets from a handoff artifact, and initializer/worker agent pairs specifically to keep an agent coherent across sessions that exceed a single context window - none of which are prompt-level concerns, all of which are harness-level engineering.

Loop Engineering: Governing Iteration Inside the Harness

Loop engineering lives inside the harness and governs a narrower question: given that the model can call tools repeatedly, what decides when it calls another tool, stops, or hands control back to a human? The canonical shape of this loop, formalized in the ReAct pattern from Yao et al., is: the model reasons about the current state, takes an action, observes the result, and repeats. What ReAct's original formulation does not specify - and what production systems have to add - is the governance around that repetition: a maximum iteration count, a definition of "done" that does not rely purely on the model's self-report, and an error-handling policy that distinguishes a transient failure worth retrying from a structural one that should halt the run. This is the layer where most of the reliability work described in Dex Horthy's widely referenced 12-Factor Agents principles actually lives.

Graph Engineering: Composing Agents and Steps Into a Workflow

Graph engineering is the layer above the loop, and it becomes necessary once a task naturally decomposes into distinct phases or distinct specialized agents rather than one long iterative loop. LangGraph, the most widely adopted framework in this space, models an application as a directed graph in which nodes represent an agent invocation, a tool call, or a deterministic function, and edges represent permitted transitions, some of which are conditional on the current state. This is close to what Anthropic's own "Building Effective Agents" guide describes as workflow patterns - prompt chaining, routing, parallelization, orchestrator-worker, and evaluator-optimizer - each of which is naturally expressed as a small graph shape rather than a single loop. A team at LangChain that has been building graph-based agent systems for several years has made a specific observation worth internalizing here: most production agent graphs are not true DAGs, because retries, revision cycles, and human-approval pauses all introduce cycles back into earlier nodes, which means a loop is best understood as a special case of a graph rather than a separate concept.

Implementation: Building the Three Layers in Code

Seeing these three layers as separate, composable pieces of code is more useful than seeing them as three separate philosophies. Below is a minimal but realistic implementation of each: a harness that owns tool registration and permission checks, a loop controller that governs iteration inside that harness, and a small graph executor that composes multiple agent runs - each of which could be tested, replaced, or scaled independently of the other two. This is the same principle that shows up in conventional software architecture as separation of concerns, applied to a system where one of the components is a language model instead of deterministic code. The harness layer shown first is the foundation everything else sits on, and its job is narrow by design: register tools, gate every call against a permission policy before it executes, and manage how much context accumulates. Nothing in this layer knows anything about iteration count or workflow structure - it only knows how to safely execute one tool call and how to keep a running context buffer under control.

// harness.ts - agent harness layer: tools, permissions, context management
type PermissionLevel = "auto" | "confirm" | "denied";

interface ToolDefinition {
  name: string;
  description: string;
  permission: PermissionLevel;
  execute: (input: unknown) => Promise<string>;
}

export class AgentHarness {
  private tools = new Map<string, ToolDefinition>();
  private contextBuffer: string[] = [];
  private readonly maxContextEntries: number;

  constructor(maxContextEntries = 40) {
    this.maxContextEntries = maxContextEntries;
  }

  registerTool(tool: ToolDefinition) {
    this.tools.set(tool.name, tool);
  }

  async callTool(name: string, input: unknown, requestApproval: () => Promise<boolean>) {
    const tool = this.tools.get(name);
    if (!tool) return { ok: false, error: `unknown_tool:${name}` };

    if (tool.permission === "denied") {
      return { ok: false, error: `permission_denied:${name}` };
    }
    if (tool.permission === "confirm") {
      const approved = await requestApproval();
      if (!approved) return { ok: false, error: `approval_rejected:${name}` };
    }

    const result = await tool.execute(input);
    this.appendContext(`[tool:${name}] ${result}`);
    return { ok: true, result };
  }

  private appendContext(entry: string) {
    this.contextBuffer.push(entry);
    if (this.contextBuffer.length > this.maxContextEntries) {
      // Compact rather than truncate blindly: keep the most recent entries
      // and collapse the older ones into a single summary marker.
      const overflow = this.contextBuffer.length - this.maxContextEntries;
      this.contextBuffer.splice(0, overflow, `[compacted ${overflow} earlier entries]`);
    }
  }

  getContext(): string {
    return this.contextBuffer.join("\n");
  }
}

The loop layer sits directly on top of the harness and never touches a tool or a permission check itself - it only decides whether to keep asking the model for another step, treating the harness as an opaque dependency it calls through execute_tool and harness_context. Separating this from the harness means the same harness can be reused under a completely different iteration policy without touching tool or permission code at all, and it means the loop's stopping behavior can be unit-tested against a fake harness that returns scripted tool results, without ever making a real network call or touching a real file system.

# loop_controller.py - loop engineering layer: iteration, stopping, escalation
from dataclasses import dataclass
from typing import Callable, Optional

@dataclass
class LoopResult:
    status: str  # "done" | "escalated" | "failed"
    output: Optional[str] = None
    steps_taken: int = 0

def run_agent_loop(
    call_model: Callable[[str], dict],
    harness_context: Callable[[], str],
    execute_tool: Callable[[str, dict], dict],
    max_steps: int = 8,
    max_consecutive_errors: int = 2,
) -> LoopResult:
    consecutive_errors = 0

    for step in range(1, max_steps + 1):
        decision = call_model(harness_context())

        if decision.get("action") == "final_answer":
            return LoopResult(status="done", output=decision.get("answer"), steps_taken=step)

        if decision.get("action") == "request_human":
            return LoopResult(status="escalated", output=decision.get("reason"), steps_taken=step)

        if decision.get("action") == "tool_call":
            result = execute_tool(decision["tool_name"], decision.get("tool_input", {}))
            if not result.get("ok"):
                consecutive_errors += 1
                if consecutive_errors >= max_consecutive_errors:
                    return LoopResult(status="failed", output=result.get("error"), steps_taken=step)
                continue
            consecutive_errors = 0
            continue

        # Malformed or unrecognized action - treat as a recoverable error, not a crash.
        consecutive_errors += 1
        if consecutive_errors >= max_consecutive_errors:
            return LoopResult(status="failed", output="unrecognized_action", steps_taken=step)

    return LoopResult(status="failed", output="max_steps_reached", steps_taken=max_steps)

The graph layer treats a fully assembled harness-plus-loop as a single node, and its own job is limited to deciding which node runs next given the current state. The example below sketches a research-draft-verify workflow with a conditional edge that routes back to the draft step when verification fails, which is precisely the kind of cycle that makes real agent graphs different from pure DAGs - the verify node's router can send execution backward to draft an arbitrary number of times before ever reaching __end__, and the max_transitions guard exists for exactly the same reason the loop controller above has a max_steps guard: an external, code-level limit that does not depend on any single node behaving well.

# graph_executor.py - graph engineering layer: nodes, edges, state, cycles
from typing import Callable, Dict, Any

class AgentGraph:
    def __init__(self):
        self.nodes: Dict[str, Callable[[dict], dict]] = {}
        self.edges: Dict[str, Callable[[dict], str]] = {}  # node_name -> router

    def add_node(self, name: str, fn: Callable[[dict], dict]):
        self.nodes[name] = fn

    def add_conditional_edge(self, from_node: str, router: Callable[[dict], str]):
        self.edges[from_node] = router

    def run(self, start_node: str, state: dict, max_transitions: int = 20) -> dict:
        current = start_node
        for _ in range(max_transitions):
            state = self.nodes[current](state)
            if current not in self.edges:
                return state  # terminal node
            current = self.edges[current](state)
            if current == "__end__":
                return state
        raise RuntimeError("graph exceeded max_transitions without reaching an end state")

# Wiring: research -> draft -> verify -> (revise draft | end)
graph = AgentGraph()
graph.add_node("research", lambda s: {**s, "notes": gather_research(s["topic"])})
graph.add_node("draft", lambda s: {**s, "draft": write_draft(s["notes"])})
graph.add_node("verify", lambda s: {**s, "verified": run_fact_check(s["draft"])})

graph.add_conditional_edge("research", lambda s: "draft")
graph.add_conditional_edge("draft", lambda s: "verify")
graph.add_conditional_edge(
    "verify", lambda s: "__end__" if s["verified"]["passed"] else "draft"
)

Trade-offs and Common Pitfalls

The most common harness-level failure is scope creep in the tool surface: every new capability the agent needs gets added as another tool with its own permission rules, until the model is choosing between forty loosely documented functions and the harness's permission logic has become an unreviewable tangle of special cases. The fix is not a smarter model - it is treating the tool surface the way a well-run team treats a public API: versioned, minimal, and reviewed for overlap before a new tool is added rather than after three overlapping ones already exist. A related and more dangerous failure is granting broad, auto-approved permissions to move fast during prototyping and then shipping that configuration to production unchanged, which converts a convenience during development into a standing security liability once the harness is handling real user data or real financial actions.

Loop-level pitfalls tend to be about missing or soft stopping conditions. A loop that lets the model decide unilaterally when a task is complete will occasionally keep going well past the point of diminishing returns, burning tokens and latency on a task that was effectively finished several steps earlier, and will occasionally stop early on a task that was not actually complete, because the model's self-assessment is not a reliable substitute for an external check. Graph-level pitfalls are different in character: teams that adopt a graph framework before they need one often end up encoding logic that would be simpler as a single well-governed loop, adding transition complexity that has to be maintained without adding any real flexibility, since - as practitioners building graph-based systems for several years now point out - a loop is already a graph with one node, and reaching for a full multi-node graph before a task actually decomposes into distinct phases is premature structure.

Best Practices

At the harness layer, the most durable habit is treating permissions as a first-class design decision made up front rather than a checklist item added during a security review. Every tool should declare its permission tier explicitly - auto-approved, requires confirmation, or forbidden - and that tier should be reviewed whenever the tool's underlying capability changes, not just when the tool is first added. Context management deserves the same explicitness: decide in advance whether long-running sessions will use periodic compaction, a hard reset with a handoff artifact, or a hybrid of the two, rather than letting the context buffer grow until it silently degrades response quality.

At the loop layer, the practice that prevents the largest share of production incidents is making the stopping condition external to the model: a maximum step count enforced in code, a definition of task completion that can be checked against structured output rather than inferred from free text, and an explicit escalation path for a human when the loop cannot make progress. Logging every step of the loop - the model's decision, the tool called, the result, and the running error count - turns an agent failure from a mystery into a replayable incident, which matters enormously once an agent is handling requests a developer did not personally originate.

At the graph layer, the practice worth adopting early is designing state as an explicit, typed structure that passes between nodes rather than an implicit accumulation of conversation history. A typed state object makes it possible to unit-test an individual node in isolation, to add a new conditional edge without re-deriving what information is available at that point in the workflow, and to visualize the graph as documentation rather than as something only fully understood by whoever wrote it. Keeping the number of distinct node types small - deterministic function, tool call, agent invocation, human checkpoint - also keeps a graph legible as it grows, which matters more than it sounds, since an illegible graph is nearly as hard to debug as an unstructured loop.

Mental Models for Thinking in Layers

A useful way to hold these three layers in mind is the metaphor of a single worker, a shift, and an organization chart.

  • The harness is the workshop the worker operates in: which tools are on the bench, which ones are locked in a cabinet requiring a supervisor's sign-off, and what reference material is kept within arm's reach versus filed away.
  • The loop is that worker's shift: how many tasks they attempt before checking in, what counts as finishing a task versus abandoning it, and what they do when a tool breaks in their hands.
  • The graph is the organization chart above any single worker: which specialist handles research, which handles drafting, which handles review, and the explicit handoff rules between them - including the entirely normal case where a reviewer sends work back to the drafter rather than every task flowing in one direction.

A second, more technical analogy maps cleanly onto ideas most backend engineers already have: the harness is the runtime and sandbox a service executes in - its permissions, its resource limits, its available libraries. The loop is the retry-and-backoff logic inside a single service call - how many attempts, what counts as success, when to give up and raise an alert. The graph is the orchestration layer above many services - a workflow engine or a state machine coordinating calls between them, complete with the same conditional branches and occasional cycles a real business process requires. Nothing about adding a language model into one of these components changes the underlying discipline; it changes what has to go into the harness, because a model, unlike a conventional service, cannot be trusted to enforce its own permission boundaries.

The 80/20 Insight

Of the three layers, the harness's permission model returns the most safety and reliability per hour invested, because a poorly scoped permission model is the one class of failure that can turn a bug into an incident with real-world consequences - a wrongly executed database write, a sent email that should have required approval, an API call against production instead of staging. Getting this right does not require sophisticated engineering: it requires an explicit, reviewed permission tier on every tool and a default posture of requiring confirmation for anything irreversible, which is a policy decision more than a technical one, and one that pays for itself the first time it prevents a single bad action rather than merely a wrong answer.

The second highest-leverage investment is an externally enforced loop-stopping condition, for the same reason it mattered in isolation: a runaway loop is a cost and safety problem independent of how good the underlying model is, and it is prevented entirely by a handful of lines of control-flow code rather than by any amount of additional prompting. Together, a reviewed permission model and a hard-capped loop eliminate the two failure modes most likely to page someone at 3 a.m. - an agent doing something it should not have been allowed to do, and an agent doing something indefinitely that it should have stopped doing.

Graph structure, by contrast, is the layer to under-invest in until the application genuinely demands it. A single well-governed loop inside a well-scoped harness handles a large share of real agent use cases, and the return on formal graph structure only appears once a task decomposes into distinct phases with different context needs, different tools, or different agents - at which point the explicit structure pays for itself in debuggability. Reaching for a graph framework before that point mostly adds surface area to maintain without adding capability the simpler loop did not already have.

Key Takeaways

These three layers compound rather than substitute for each other: a well-designed graph built on top of an unsafe harness inherits that harness's safety problems, and a tightly governed loop cannot compensate for a permission model that lets the model take actions it should never have been allowed to attempt. Treat each layer as independently reviewable, independently testable, and independently owned, and use the list below as a starting checklist the next time an agent misbehaves in a way that is not obviously a model-quality problem.

  • Give every tool in the harness an explicit permission tier, and default new or uncertain tools to requiring confirmation rather than auto-approval.
  • Never let the model be the sole judge of when a loop is finished; enforce a maximum step count and a structurally checkable definition of "done" in code.
  • Log every step of the loop - decision, action, result, error count - so a failure can be replayed rather than only reported.
  • Reach for graph structure only once a task genuinely decomposes into distinct phases or specialized agents; a single governed loop is often sufficient and is easier to reason about.
  • Design graph state as an explicit, typed object rather than an implicit conversation history, so individual nodes remain testable in isolation.

Applied together, these five habits map directly onto the three-layer model: the first is a harness discipline, the second and third are loop disciplines, and the fourth and fifth are graph disciplines. A team that adopts all five will spend noticeably less time debugging agent incidents by guesswork, because each failure now has an obvious layer to start investigating.

Conclusion

Agent harness engineering, loop engineering, and graph engineering describe three real, separable layers of responsibility that sit between a language model and a working production system. The harness decides what the model is allowed to touch and how its working context is managed. The loop decides how many times the model gets to try, and what ends the attempt. The graph decides how multiple agents, tools, and checkpoints relate to each other once a single loop is no longer an adequate description of the work. Treating these as one undifferentiated blob of "agent engineering" is what leads teams to fix a permissions bug by editing a prompt, or to reach for a full orchestration framework to solve a problem that a hard iteration cap would have solved just as well.

What makes this framing useful in practice is that it gives a team a diagnostic order to work through when something goes wrong, rather than a single, undifferentiated impulse to reword the system prompt. A model taking an action it should not have taken is a harness problem. A model that never stops, or stops too early, is a loop problem. A workflow that is hard to reason about because responsibilities are smeared across one enormous prompt is a graph problem waiting to be made explicit. None of these three diagnoses require a research breakthrough to fix - they require the same discipline that has always separated reliable software from fragile software, applied deliberately to a component whose output happens to be generated by a language model rather than written by hand.

As the underlying models keep improving, it is worth watching which of these three layers shrinks and which grows, because the trend so far has not been uniform. Loop governance has become somewhat lighter as models get better at recognizing their own uncertainty and asking for help rather than guessing, but harness design has if anything grown more important, since a more capable model is also a model more likely to attempt something ambitious with whatever access it has been granted. Graph structure sits in between: as agents themselves get more reliable, some work that once required an explicit multi-node workflow can collapse back into a single well-harnessed loop, while genuinely multi-phase or multi-team work keeps the graph layer relevant regardless of how good any individual agent becomes.

References