What is an agent harness? The loop, tools and context around the model

An agent harness is the software around a language model that turns it into an agent: the loop that calls the model and runs the tools it asks for, the context assembled for each call, memory, permissions, the sandbox the commands run in, retries, and the checks that decide when a task is done. Anthropic’s own definition is short: “the software scaffolding around a model: the loop, tools, context management, and guardrails that turn raw intelligence into a working agent”.

The term matters in 2026 because the evidence has piled up that the harness moves results about as much as the model does. The same model scores very differently on agent benchmarks depending on what surrounds it, and the most capable coding products, Claude Code and Codex, are now described by their makers as harnesses first. This explainer is for engineers building or buying agents. It covers what a harness contains, what one turn of the loop looks like, the evidence for the “harness matters as much as the model” view, why context engineering is the hardest part, and when to build versus adopt.

For how the model itself spends tokens on reasoning, see our explainer on what reasoning models actually do. This piece is about everything outside the weights.

What an agent harness is, and what it is not

An agent harness is the runtime that sits between a model and the world. The model receives text and returns text, possibly a structured request to call a tool. The harness does everything else: it builds the prompt, executes the requested tool, feeds the result back, decides whether to continue, and enforces what the agent is allowed to do.

The word came into wide use through the companies building the best-known agents. OpenAI’s January 2026 post Unrolling the Codex agent loop describes “the Codex harness, which provides the core agent loop and execution logic that underlies all Codex experiences”, and calls the agent itself “(or ‘harness’)”. Anthropic’s Claude Agent SDK announcement calls it “the agent harness that powers Claude Code”, renamed from the Claude Code SDK because the same harness can power other kinds of agents. A June 2026 survey on arXiv, From Question Answering to Task Completion, frames an LLM agent as “a foundation model coupled with an execution harness” and splits the harness into six runtime responsibilities: observation, context, control, action, state and verification.

Three neighbouring terms cause confusion:

The boundary between framework and harness is blurry, and some products are both. The OpenAI Agents SDK describes itself as a small set of primitives, yet it now ships sandbox agents, sessions and tracing, which are harness features. The useful distinction is who has made the decisions: a framework leaves the loop, the tool set and the context policy to you; a harness has made them, and you customise from there. Our survey of agent frameworks maps the framework camps in more detail.

The simplest possible harness is small. mini-SWE-agent, maintained by the SWE-bench team, is “just some 100 lines of python” for the agent class, gives the model nothing but a shell, runs each action with subprocess.run, and keeps “a completely linear history” where every step just appends to the messages. Its README says it scores above 74% on SWE-bench Verified. Production harnesses are far larger because they add everything the minimal one leaves out: permissions, sandboxing, compaction, memory, subagents and recovery.

The anatomy of an agent harness

A production agent harness does five jobs on every turn: it runs the loop, curates context, exposes tools, enforces safety, and keeps state while checking the work. Each job has a small number of standard techniques, and most of the differences between harnesses come from which techniques they pick and how aggressively they apply them.

Figure 1: The weights are one component. Everything in these five columns is harness, and each item is a design decision that changes results.

The loop. The core is a while-loop: send the prompt, read the response, and if the model asked for a tool, run it and go round again. Codex ends a turn when the model returns an assistant message rather than a tool call. Around that sit budgets (maximum turns, wall-clock limits), retry logic for failed model calls, and detectors for unproductive behaviour. LangChain’s harness, for example, has a LoopDetectionMiddleware that counts edits per file and nudges the agent to reconsider after a threshold.

Context. The harness decides what the model sees: the system prompt, project instructions such as AGENTS.md or CLAUDE.md, tool definitions, retrieved files, prior turns and tool results. Because the model is stateless, the harness resends all of it on every call. This is where context engineering lives, and it gets its own section below.

Tools. The actions available to the model: a shell, a file editor, search, browser automation, Model Context Protocol (MCP) servers, and the ability to spawn subagents. Tool design is a harness decision with measurable effects. Anthropic reported in late 2024 that Claude 3.5 Sonnet reached 49% on SWE-bench Verified, then state of the art, with only a bash tool and a text editor tool.

Safety. Permission rules that decide which tool calls run automatically and which need a human, plus a sandbox that fences the filesystem and network so that a mistaken or prompt-injected command cannot do much damage. Hooks let teams run their own code before or after any tool call.

State and checks. Memory that outlives a context window (notes files, progress logs, git history), and verification: running the tests, a separate evaluator agent, or a checklist pass before the agent may declare victory. Tracing belongs here too, because a harness that cannot be replayed cannot be debugged.

The Claude Agent SDK’s capability list is a good concrete inventory: built-in tools, hooks, subagents, MCP, permissions, sessions, skills, commands and memory, and plugins. The OpenAI Agents SDK lists agents with a built-in loop, sandbox agents, handoffs, guardrails, function tools, MCP server tool calling, sessions, human-in-the-loop and tracing. The vocabularies differ; the columns in Figure 1 are the same.

One turn of the agent loop, step by step

One turn of an agent loop has six steps: assemble the context, call the model, check whether it asked for a tool or answered, check the tool call against policy, run it in the sandbox, and append the result, compacting if the context is near its limit. The model is involved in exactly one of those steps.

OpenAI’s Codex post is the most detailed public description of these mechanics, so it is worth following.

1. Assemble the context. Codex builds an input list for the Responses API: a developer message describing the sandbox that applies to its shell tool, project instructions, environment details, the tool list, then the user’s message. Static content goes first.
2. Call the model. The request goes to the inference API. In a production harness this call is also where rate limits, provider errors and retries are handled.
3. Tool call or final answer. If the model returns an assistant message, the turn ends and control returns to the user. If it returns a tool call, the loop continues.
4. Check policy. The harness decides whether this call may run unattended. Codex’s approvals documentation describes an approval policy that “controls when it must stop and ask you before acting”.
5. Run in the sandbox. The command executes inside the fenced environment.
6. Append the result. The output is appended to input and the model is called again.

Two design details from the Codex post show how much engineering hides in these steps. First, the loop is quadratic in the amount of JSON sent, since each call resends the whole conversation. Codex accepts that rather than using server-side conversation state, “primarily to keep requests fully stateless and to support Zero Data Retention (ZDR) configurations”, and recovers the cost through prompt caching: because the old prompt is “an exact prefix” of the new one, cache hits make sampling “linear rather than quadratic”. Second, anything that changes the prefix breaks the cache. Codex’s early MCP support had a bug where it “failed to enumerate the tools in a consistent order, causing cache misses”, and mid-session configuration changes are appended as new messages rather than edited into old ones.

Figure 2: The model makes one decision per turn. Retries, policy, sandboxing and compaction are all harness work, and each branch is a place where two harnesses running the same model diverge.

Why the harness matters as much as the model

The 2026 consensus that the harness matters as much as the model rests on controlled comparisons: hold the model fixed, change the harness, and scores move by amounts that used to require a new model generation. Several of these comparisons are now public, and the leading benchmarks have started reporting the harness beside the model for that reason.

Same model, different harness

The cleanest example is LangChain’s February 2026 post, Improving Deep Agents with harness engineering. Its coding agent rose from 52.8% to 66.5% on Terminal-Bench 2.0, “from Top 30 to Top 5”, with the model fixed at GPT-5.2-Codex. The team limited itself to three knobs: the system prompt, the tools, and middleware (hooks around model and tool calls). The changes were unglamorous: prompting the agent to build and then verify, a PreCompletionChecklistMiddleware that forces a verification pass before exit, a LocalContextMiddleware that maps the working directory at start-up, time-budget warnings, and loop detection.

Even reasoning effort turned out to be a harness decision. Running at the model’s highest setting throughout scored 53.9%, mostly because of agent timeouts; running at “high” scored 63.6%. The best result came from what the post calls a “reasoning sandwich”: maximum effort for planning and final verification, less in the middle. The same post reports Claude Opus 4.6 at 59.6% on an earlier version of the harness, a reminder that harness improvements are often tuned to one model.

Anthropic reports the same pattern on non-coding work. In an April 2026 article on agent harness design, giving Opus 4.6 the ability to filter its own tool outputs with code raised BrowseComp accuracy from 45.3% to 61.6%; giving Sonnet 4.5 a memory folder lifted BrowseComp-Plus from 60.4% to 67.2%. With the same compaction setup, Sonnet 4.5 stayed flat at 43% while Opus 4.5 scaled to 68% and Opus 4.6 to 84%, which shows the coupling runs both ways: some harness features only pay off with models trained to use them.

Benchmarks now rank model and harness as a pair

The Terminal-Bench 2.0 leaderboard lists an “Agent” column beside every model. As of mid-October 2026 the top entry is Claude Opus 5.5 running in Claude Code at 64.8% (±3.1), and the table mixes harnesses: Codex for OpenAI’s models, Claude Code also for third-party models such as GLM-5.3, Grok Build for xAI’s, mini-SWE-agent for Gemini 3.8 Flash. A score on that board describes a model-harness pair, and comparing two rows that use different agents compares two variables at once.

SWE-bench took the opposite approach to isolate the model. Its “bash only” leaderboard runs every model inside the same minimal harness, mini-SWE-agent, so that the model is the only thing that changes. The result is informative in its own right: with the harness held constant, Claude 4.5 Opus at high effort scored 76.8% at an average of $0.75 per task, while Gemini 3 Flash and MiniMax M2.5 both scored 75.8% at $0.36 and $0.07. When the harness is fixed, frontier models cluster tightly and the differences show up in cost.

Model cards have adjusted accordingly. As our guide to reading a model card notes, vendors now state which harness produced each agentic score, and Anthropic has switched harnesses for a benchmark because one of them timed out far more often at high effort. A benchmark number without a harness is no longer a complete claim.

The harness also decides the bill

Anthropic’s March 2026 post on harness design for long-running apps compared a single agent with a three-agent planner, generator and evaluator harness on the same one-sentence prompt for a game-building application. The solo run took 20 minutes and cost $9; the full harness ran for six hours and cost $200. The author says the harness was “over 20x more expensive” but that the difference in quality was “immediately apparent”: in the solo run, the central feature of the application did not work.

Anthropic’s multi-agent research post puts numbers on the trade. Its multi-agent configuration beat a single agent by 90.2% on an internal research evaluation, but agents typically use about four times as many tokens as chat, and multi-agent systems about fifteen times. In its BrowseComp analysis, token usage alone explained 80% of the variance in performance. A harness is partly a policy for how many tokens to spend, and on what.

OpenAI’s version: humans steer, agents execute

OpenAI’s February 2026 post Harness engineering gave the practice its name. A team built an internal product over five months with “0 lines of manually-written code”: roughly a million lines and about 1,500 merged pull requests, starting with three engineers driving Codex, an average of 3.5 pull requests per engineer per day. The lessons are almost all harness lessons. When something failed, the team asked “what capability is missing, and how do we make it both legible and enforceable for the agent?” Answers included a short AGENTS.md of roughly 100 lines used as “a map, not a 1,000-page instruction manual”, custom linters whose error messages inject remediation instructions into the agent’s context, a per-worktree observability stack the agent can query with LogQL and PromQL, and recurring “doc-gardening” agents that fix stale documentation.

Context engineering is the harness’s hardest job

Context engineering is the practice of deciding what enters the model’s context window on each turn of the loop. Anthropic’s guide defines good context engineering as “finding the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome”. Unlike prompt writing, it is not a one-off task: the curation “happens each time we decide what to pass to the model”.

The reason it matters is that long context degrades. Chroma’s July 2025 study, Context rot, tested 18 models and found performance falling as input length grew, “often in surprising and non-uniform ways”, even on simple tasks. Anthropic frames this as an “attention budget” that every token depletes. A harness that stuffs every file, every tool result and every past turn into the window gets a slower, more expensive and less accurate agent.

Five techniques account for most of the practice:

Compaction is the first lever. Codex originally required a manual /compact command; it now calls a dedicated compaction endpoint automatically when the conversation exceeds an auto_compact_limit, and the result includes an encrypted item that “preserves the model’s latent understanding of the original conversation”. Anthropic warns that “overly aggressive compaction can result in the loss of subtle but critical context”, and recommends tuning the compaction prompt first for recall, then for precision. Subagents trade the other way: each may use “tens of thousands of tokens or more” but returns “a condensed, distilled summary of its work (often 1,000-2,000 tokens)”.

Context engineering also has a cost dimension that is easy to miss. Manus, in its context engineering write-up, called the KV-cache hit rate “the single most important metric for a production-stage AI agent”, because its agents average around 100 input tokens for every output token and about 50 tool calls per task. Cached input is far cheaper than uncached: Anthropic’s harness article notes that cached tokens cost 10% of base input tokens. That is why harnesses keep static content first, append changes as new messages instead of editing old ones, avoid switching models mid-session, and avoid adding or removing tools, which sit in the cached prefix. Manus goes as far as masking tool choices at decoding time rather than removing tools from the prompt. It also reports having “rebuilt our agent framework four times”, which is a fair summary of how empirical the discipline still is.

The newest move is to hand some context decisions back to the model. Anthropic’s harness design article argues that “agent harnesses encode assumptions about what Claude can’t do on its own”, and that those assumptions go stale. Skills, whose short descriptions sit in context while the full instructions load only when needed, let the model assemble its own context; code execution lets it filter tool results before they reach the window. The harness still sets the boundaries; the model fills in more of the detail.

Permissions and the agent sandbox

An agent sandbox is the isolated environment in which an agent’s commands run, so that an agent that makes a mistake or follows injected instructions cannot read secrets, damage the host or send data out. Permissions decide which actions need approval; the sandbox limits the damage of the actions that run without it. Good harnesses use both.

Anthropic’s sandboxing post is explicit that both layers of isolation are needed: “Without network isolation, a compromised agent could exfiltrate sensitive files like SSH keys; without filesystem isolation, a compromised agent could easily escape the sandbox and gain network access.” Claude Code enforces this with operating-system primitives, bubblewrap on Linux and seatbelt on macOS, allowing writes only in the working directory and routing network traffic through a proxy that enforces a domain allow-list. Anthropic reports that in internal use, sandboxing “safely reduces permission prompts by 84%”, which is the real payoff: a well-fenced agent can be left alone for longer.

Codex makes similar choices by default. Its documentation says the agent “runs with network access turned off”, with “write permissions limited to the active workspace”, and offers named modes from read-only to workspace-write, combined with an approval policy such as on-request. One caveat from the Codex loop post is worth repeating for anyone wiring in third-party tools: the Codex sandbox “applies only to the Codex-provided shell tool”, and tools provided by MCP servers “are not sandboxed by Codex and are responsible for enforcing their own guardrails”. Tools that reach outside the sandbox need their own policy layer.

Anthropic’s harness design article adds a design rule that is easy to apply: promote risky actions to dedicated, typed tools. A bash tool gives the harness only a command string; a dedicated tool gives it “typed arguments it can intercept, gate, render, or audit”. Reversibility is a good test. Reading a file can run unattended; an external API call that moves money should be its own tool with a confirmation step.

Hosted harnesses move the sandbox off the developer’s machine. Claude Managed Agents describes itself as a “pre-built, configurable agent harness that runs in managed infrastructure”, with sessions in an Anthropic-managed cloud sandbox or a self-hosted sandbox; the OpenAI Agents SDK’s sandbox agents run specialists “inside real isolated workspaces” and support resumable sandbox sessions.

Memory, state and verification on long tasks

Long-running agents fail in predictable ways, and the fixes are almost entirely harness work: structured state that survives a context reset, a clear definition of done, and verification that does not rely on the agent’s own judgement.

Anthropic’s November 2025 post on effective harnesses for long-running agents lists the failures it saw when asking an agent to build a clone of a web application across many sessions: the agent tried to do too much at once and ran out of context mid-feature; later sessions saw partial progress and declared the job done; features were marked complete “without proper testing”. The fix was a two-part harness. An initializer agent runs once to create a setup script, a claude-progress.txt log, a first git commit and a feature list of more than 200 items, all marked failing. A coding agent then works one feature at a time and leaves structured updates. The feature list is JSON rather than Markdown because the model was less likely to overwrite it inappropriately, and coding agents may only change each item’s passes field.

The follow-up work showed how harnesses must change as models do. Sonnet 4.5 showed what Anthropic calls “context anxiety”, wrapping up work early as it approached what it believed was its context limit, so the earlier harness used full context resets with a handoff file. With Opus 4.5 that behaviour “largely” disappeared, and the later harness dropped resets entirely in favour of automatic compaction. A harness feature that was essential for one model became overhead for the next.

Verification is where the biggest quality gains came from. Once the agent was explicitly told to test “as a human user would” with browser automation, performance “dramatically improved”. In the three-agent harness, a separate evaluator clicked through the running application with the Playwright MCP server and graded each sprint against criteria with hard thresholds. Anthropic is candid that “out of the box, Claude is a poor QA agent”, one that found real bugs and then “talk[ed] itself into deciding they weren’t a big deal”. Tuning a skeptical external evaluator proved “far more tractable than making a generator critical of its own work”. The general principle is the same as LangChain’s pre-completion checklist: do not let the agent that did the work be the only judge of whether it is finished.

The model-access layer: where an AI gateway fits

Every harness has a model-access layer: the code that sends each turn to a provider, handles rate limits and errors, chooses a model, records cost and traces the call. In a single agent this lives in the SDK client. In an organisation running several harnesses, it is the layer most often pulled out into a shared AI gateway, so that keys, failover, budgets and logs are configured once rather than in every agent.

The case is practical. An agent makes dozens to hundreds of model calls per task, so transient provider errors that a chat application rarely sees become routine. Different teams run different harnesses, Claude Code here, Codex there, a custom agent built on a framework somewhere else, and each would otherwise hold its own provider keys and its own retry logic. Spend is hard to attribute when every harness bills to one account. A gateway also gives the “traces of every call” column in Figure 1 a single place to live, regardless of which harness made the call.

Bifrost, an open-source AI gateway written in Go by Maxim AI, is a useful worked example because its documentation addresses agent harnesses directly. Its CLI agents guide exposes OpenAI-, Anthropic- and Gemini-compatible endpoints so that Claude Code, Codex CLI, Gemini CLI and other coding agents can be pointed at it, and describes a CLI that launches those agents “without editing each harness’s configuration manually”. Its retries and fallbacks documentation shows the kind of logic a harness would otherwise reimplement: transient 5xx errors retry the same key with exponential backoff and jitter, a 429 rotates to another key but still backs off because quotas are often shared at account level, a rejected or exhausted key is marked dead for the rest of the request, and when a provider’s retries are spent the request moves to the next provider in a fallback chain. Each failed attempt is recorded with a failure class on an attempt trail.

The same layer can carry spending and tool policy. Bifrost’s budgets are hierarchical across customers, teams and virtual keys, so each agent or team can be given its own key and cap. Its traces follow the OpenTelemetry GenAI conventions, with span names such as gen_ai.chat, so model calls line up with the rest of a team’s telemetry. For tools, Bifrost acts as an MCP gateway, connecting to MCP servers and filtering which tools each virtual key may use. Its code mode, in which the model writes Python to orchestrate tools instead of loading every tool definition, is a context-engineering technique moved into the gateway: on its published test with 508 tools across 16 servers, input tokens fell by 92.8%. Bifrost’s published benchmark reports 11 µs of added overhead per request at 5,000 requests per second.

A gateway does not replace the harness. It does not decide what goes into the context, run the sandbox or check whether the task is done. It takes over the edge where the harness meets model providers and remote tools, which is exactly the part that should be shared across harnesses rather than rebuilt in each. Our explainer on AI gateways covers that request path in detail.

Build or adopt an agent harness

Most teams should adopt an existing agent harness and customise it, not write a loop from scratch. The loop itself is easy; the years of fixes for caching, compaction, permissions, sandboxing and recovery are not. Build your own only when traces show the adopted harness is what limits results, and even then, borrow its parts.

Figure 3: The choice is rarely binary. Teams usually adopt a harness for the loop and sandbox, customise prompts, tools and hooks, and share the model-access layer across harnesses.

The options in October 2026 fall into four groups:

Anthropic’s own advice has been consistent since its 2024 guide Building effective agents: find “the simplest solution possible”, and use frameworks to start quickly but “don’t hesitate to reduce abstraction layers and build with basic components as you move to production”. The 2026 version adds a corollary: whatever you build, plan to remove parts of it as models improve. Our ranking of agent orchestration frameworks covers the framework options for teams taking that route.

A worked example

Consider a team building an internal agent that triages failing CI runs and opens fix pull requests. The work is file and shell work, so the obvious start is a coding harness through its SDK rather than a new loop. The team’s real effort then goes into harness customisation: a short instructions file that maps the repository, a dedicated open_pull_request tool with a confirmation hook instead of raw git push through bash, a sandbox that allows writes only to the checkout and network access only to the package registry, and a verification step that reruns the failing job before the agent may finish. Model access goes through the company’s gateway with its own virtual key and monthly budget, so the agent’s spend and every call it makes show up alongside the other teams’ agents. None of this requires writing a loop.

The limits of the evidence

The “harness matters as much as the model” claim is well supported, but three cautions apply. First, most of the evidence is published by the companies selling harnesses or models. LangChain’s 13.7-point gain is its own measurement on its own agent, and Terminal-Bench reports confidence intervals of around three points per entry, so small harness changes can fall inside the noise. Our editorial on benchmarks as marketing covers the general problem.

Second, harness gains are often model-specific. LangChain’s improvements were tuned on GPT-5.2-Codex; Anthropic’s compaction results scaled with some models and stayed flat with another; context resets helped Sonnet 4.5 and became unnecessary with Opus 4.5. A harness tuned for this year’s model can carry dead weight for next year’s.

Third, the trend points towards thinner harnesses for some jobs. The Anthropic harness article’s central advice is to “ask what you can stop doing” as models improve: let the model orchestrate tools in code, manage its own context with skills, and choose what to persist. The SWE-bench “bash only” results show frontier models within a point or two of each other inside a 100-line harness. What moves into the model, though, tends to be the clever parts. The boring parts, permissions, sandboxing, budgets, tracing, retries and verification, are about trust and operations rather than intelligence, and they are not going away.

What to do with this

The agent harness is now a first-class part of any AI system that acts. Treat it with the same seriousness as model selection:

• Report model and harness together. When comparing models for an agent, hold the harness fixed, and when quoting a benchmark, name the harness and effort setting.
• Adopt before you build. Start from the closest existing harness and customise prompts, tools, hooks and permissions. Write a loop only when traces show the harness is the bottleneck.
• Engineer the context deliberately. Keep static content first for caching, retrieve just in time, compact before the window fills, and give long tasks a progress file and a feature list.
• Fence the agent, then trust it more. Restrict filesystem and network together, promote irreversible actions to typed tools with confirmation, and use the reduced prompting to let the agent run longer.
• Do not let the agent grade itself. Add a verification step, tests, a checklist pass or a separate evaluator, before the agent may finish.
• Share the model-access layer. Put provider keys, failover, budgets, tracing and MCP tool policy in one gateway so that every harness the organisation runs gets them once.
• Revisit the harness with every model upgrade. Some of what you built to compensate for the last model will be overhead for the next one.

The model sets the ceiling on what an agent can do. The harness decides how close the agent gets to it, what it costs on the way, and whether anyone can trust the result.