Agent Harnesses

What Is a Harness?

The runtime orchestration layer that wraps a reasoning loop and coordinates:

• Tool execution — file editing, bash, web search, MCP servers
• Context management — what the model sees, when, and how much
• Safety enforcement — permissions, dangerous command detection, approval flows
• Session persistence — memory, state, conversation history
• Channel integration — how humans interact (CLI, Telegram, Discord, web)

From @vtrivedy10

Per the arXiv formalization (2603.05344):

• Scaffolding = constructing the agent BEFORE the first prompt (CLAUDE.md, skills, rules, context files)
• Harness = everything that happens AFTER (tool execution, context management, safety, persistence)

The moat is the harness, not the model.

“Fat Skills, Thin Harness, Deterministic App”

From @GarryTan

Why the Harness Matters More Than the Model

Empirical evidence

LangChain’s coding agent: 52.8% → 66.5% on Terminal Bench 2.0 by changing only the harness. Same model, same prompts. The 14-point jump came entirely from orchestration changes.

Market confirmation

OpenAI open-sourced codex-plugin-cc (March 30, 2026) to invoke Codex from inside Claude Code. If models were the moat, this wouldn’t exist. Models are swappable plugins. The harness is the platform.

The Claude Code leak

When Anthropic accidentally published Claude Code’s full source (512K lines TypeScript), the community ignored the model integration and reverse-engineered the 12-layer harness system, 40+ tools, and orchestration patterns. Claw Code replicated the architecture in Python overnight. 100K GitHub stars in 24 hours.

Program Design Is the Missing Layer Between Architecture and Implementation supplies the practical response for substantial changes: settle types, signatures, layout, call structure, and vertical slices before the harness begins implementation.

The Two Concepts That Matter Most

1. Context Hygiene (from Dex Horthy - Context Engineering for AI Coding Workflows)

The “dumb zone”: past ~40% context window fill, output quality drops. Everything after that gets worse.

Worst context (ranked):

1. Incorrect information (worst)
2. Missing information
3. Noise

Trajectory matters. If an agent keeps getting corrected, the conversation history trains the next token toward more mistakes. Errors compound.

The fix: compact early and often. When context fills or goes sideways, compress state to a markdown summary and start fresh. Use sub-agents for context isolation (dispatch to read a large area, return one sentence). Don’t use them as role analogues.

2. Research → Plan → Implement

Context hygiene produces a natural workflow:

1. Research — read the actual codebase, compress into truthful snapshot
2. Plan — exact file names, line snippets, test steps. “The dumbest model won’t screw this up.”
3. Implement — follow the plan with deliberately low context

This maps directly to Spec-Driven Development and AI-Native SDLC - 2026 Analysis — specs are the plan artifact, and the harness executes against them.

Architectural Philosophies

Gateway (OpenClaw, 345K stars)

A persistent process managing routing, permissions, channels, skills, and connections. The model is one of many tools it orchestrates. 15+ platform integrations. Skills marketplace.

Memory-First (Hermes, 21.6K stars)

An agent that knows your preferences, writes its own skills, and retains context across sessions beats one that’s well-routed but forgetful. Honcho user modeling, self-improving skill library.

Minimal (Pi)

A tiny extensible harness you adapt to your workflows. No opinions, no batteries — TypeScript extensions, skills, templates. Created by Armin Ronacher. For people who want to understand and control every layer.

Orchestration (Paperclip, 43.9K stars)

The harness IS the company — org charts, budgets, governance, multiple heterogeneous agents as “employees.” Not a replacement for coding harnesses; a layer above them.

Clean-Room (Claw Code, 114K stars)

Replicate Claude Code’s harness in open source. Python/Rust rewrite, model-agnostic, independently audited.

Proxy (Claude Code Router, 31.1K stars)

Intercept the harness instead of replacing it. Route different task types to different models (cheap for simple, expensive for complex). Zero migration, full compatibility.

The Current Landscape (April 2026)

Recent Events (March-April 2026)

• March 31: Claude Code source leak → Claw Code born, 100K stars in 24 hours
• April: Anthropic ships Claude Code Channels (native Telegram/Discord)
• April: Anthropic cuts off OpenClaw from Claude subscriptions
• March 30: OpenAI open-sources codex-plugin-cc (invoke Codex from Claude Code)
• March 11: Gemini CLI ships Plan Mode

Skills as the Knowledge Layer

From The Complete Guide to Building Skills for Claude - Anthropic:

Skills sit on top of the harness as the knowledge layer. The harness provides tools (what Claude can do). Skills provide workflows (how Claude should do it).

Three-level progressive disclosure:

1. YAML frontmatter — always loaded, tells Claude when to use the skill
2. SKILL.md body — loaded when relevant, full instructions
3. Linked files — discovered on demand

From Boris Cherny - Hidden Claude Code Features Thread (Claude Code lead):

• Skills + loops = powerful automation (/loop 5m /babysit)
• Hooks for deterministic lifecycle logic (SessionStart, PreToolUse, PermissionRequest, Stop)
• Worktrees for parallel execution (claude -w)
• /batch for massive parallelized changesets

What to actually do

1. Spend time on the harness, not the model. Model improvements arrive via API updates. Harness improvements require your work.
2. Watch your context window. Past 40% fill, output quality drops. Compact early, use sub-agents for isolation.
3. Write skills and context files. They encode what you’ve learned about your domain. This is what compounds.
4. Treat the knowledge graph as the durable asset. Harnesses will be replaced. Your accumulated context won’t.
5. Don’t lock into one model. Claude Code Router exists because models are swappable. The harness persists.
6. Research → Plan → Implement — the workflow that emerges from taking context seriously

2026-04 Update — Harness Governance as SpecOps

From Spec-Driven Development – Adoption at Enterprise Scale (Hari Krishnan, InfoQ, Feb 2026), cross-linked into Spec-Driven Development and AI-Native SDLC - 2026 Analysis:

• Spec authoring is a first-class engineering surface. Quality practices, version control, review processes, and continuous improvement should apply to specs the same way they apply to production code. Leigh and Ray called this “SpecOps” in their related InfoQ piece.
• Bugs are spec defects, not code defects. Two gap types: spec-to-implementation (implementation diverged from a clear spec → strengthen validation agents in the harness) and intent-to-spec (use case was missed during deliberation → improve spec elicitation). Either way, the harness itself is the thing being improved, not the individual bug.
• Quality engineering moves upstream. The job shifts from validating implementations to validating the context harnesses that guide agent execution.
• Directing agent swarms is an organizational capability, not an individual one. Adrian Cockcroft framing at QCon SF: “directing swarms of agents” is a different skill than managing individual contributors, and enterprises have to install it as a capability at team level, not at developer level.

2026-04 Update — Karpathy’s Frame on Agents, Throughput, and program.md

From Karpathy - No Priors Code Agents Autoresearch:

• Token Throughput as the New Coding Bottleneck — Dec 2024 inflection, “Peter Steinberg mode” (10 agents in parallel), orchestration as the new developer skill
• Claws - Persistent Looping Agents as App Replacement — Dobby the elf experiment, apps become APIs, agent is the customer
• Auto Research - Agents as Overnight Experimentation Engines — give an objective, a budget, and walk away; agents found hyperparameter bugs in Karpathy’s hand-tuned GPT-2 repo
• Research Org as Tunable program dot md — the research org itself becomes a markdown document you can version, A/B test, and meta-optimize
• Three Waves of AI Opportunity - Unhobbling, Physical Interface, Robotics — 1M-order-of-magnitude gap between bits and atoms; almost all 2026 value is in Wave 1

2026-04 Update — The Folder Is the Agent (44-Agent Production Setup)

From The Folder Is the Agent - Kieran Klaassen (Every, April 2026):

• The Folder Is the Agent - Context Accumulation as Specialization — an agent is not a framework or protocol. It’s a folder with CLAUDE.md + accumulated docs + skills. Change the folder, not the model, and you get a different specialist. Klaassen runs 44 agents across a handful of folders — same model (Opus 4.6), different folder = different specialist (Rails engineer vs ops engineer). Dispatch layer is a Ruby daemon with file-based messaging — no custom networking, no agent-to-agent protocol. Key principle: “Build it, use it, trust it, then orchestrate it” — don’t hand a folder to automation until you’ve used it yourself and trust its flows.

2026-04 Update — AXI: Agent-Computer Interface Design, Benchmarked

From AXI Agent eXperience Interface (Kun Chen, April 2026 — 915 benchmark runs):

• AXI Principles - Agent-Ergonomic Interface Design — 10 principles for agent-ergonomic CLI (Efficiency / Robustness / Discoverability / Fallback). Benchmarked across 915 runs: same model (Claude Sonnet 4.6), only variable is interface design, 1.6x variance in cost and duration. AXI hits 100% success at lowest cost/turns/duration on both browser and GitHub benchmarks. Mechanisms: TOON format (40% token savings over JSON), combined operations (navigate + snapshot in one call), next-step hints after output.

2026-04 Update — Agent-Native Architecture Principles

From Agent-native Architectures (Dan Shipper + Claude, Every):

• Agent-Native Architecture - Five Principles for Building After Code Ends — five principles for building apps where agents are first-class: Parity (agent matches UI), Granularity (atomic tools, outcome-based features), Composability (new features = new prompts), Emergent Capability (handles the unanticipated), Improvement Over Time (context compounds). Files as the universal agent interface. The “ultimate test”: can the agent accomplish an outcome you didn’t build a feature for?

2026-08 Update — Naming the Layers, and the Inference Service

From Your Agent Is Not the Model (Joe Wright, 2026):

The vault’s notes tend to collapse the stack into two parts — model and harness. There are four, and the missing one is the inference service: the hosted layer that runs the model and meters usage (Bedrock, Anthropic’s API), sitting between the harness and the weights.

Why the distinction earns its keep: it turns a vague symptom into a located fault. Slow or expensive is the inference layer; missing context or a malformed tool call is the harness; bad reasoning is the model or the context the harness handed it. “Same model behaves differently” is always harness, context, or tooling — never the weights.

The corollary this note already argues from the other direction: MCP and Skills are harness features. The model has no inherent knowledge of a Skill or an MCP server — the harness decides what to expose. Which is why the same Sonnet behaves unlike itself across Claude CLI, Cursor, and a LangChain app.

2026-09 Update — Three Orchestration Levels, and Routing as the Cost Lever

Claude Code now exposes orchestration at three levels, and the distinction matters because each has a different failure mode.

Reported before/after on the same job: one agent working serially takes a full day on a four-part feature, with context bloating across task switches; four agents in parallel — backend, frontend, tests, review — finish in about two hours, each with clean focused context. Second-hand, from a LinkedIn summary of an Anthropic engineer’s demo video, so the timings are illustrative rather than measured.

The cost lever is model routing by task type. Run the lead on Opus and route teammates to Sonnet at roughly 1/5 the cost. The claim is same quality for focused execution, a fraction of the spend. The underlying principle: generating code against a clear spec is cheap work; deciding what to build is not, and that is where the expensive model earns its price. This is the same lever that shows up as the single most impactful cost change in Running a Software Factory Efficiently at Uber Scale — worth reading together, since one is a demo and the other is production data.

The loop, not the prompt

Boris Cherny, Claude Code’s creator, on his own daily use: “I don’t prompt Claude anymore. I write loops and the loops do the work. My job is to write loops.” Reported second-hand via a LinkedIn post about a 30-minute walkthrough; the quote is attributed but the video was not fetched.

If accurate, it relocates the harness skill again. Prompt quality was the 2024 skill and context engineering the 2025 one. This says the 2026 unit of work is a bounded loop with an exit condition, which the human authors once and the agent executes many times. Consistent with the harness-over-model thesis at the top of this note, and with Harness Simplification as Models Improve: a loop is a smaller artifact than a scaffold.

Tool access priced per call

treg (treg.superdesign.dev) puts 2,630 tool endpoints behind one key at per-call pricing, AGPL, with credentials injected server-side rather than sitting on the agent’s machine. The interesting part is not the aggregation. It is the selection rule: the agent asks for the task, not the tool, and picks a vendor on measured success rate, speed, and time since last successful answer. Nineteen providers sell backlink data, nine sell email lookup, and the agent chooses among them on evidence rather than brand.

That is a harness-layer change disguised as a marketplace. Tool selection stops being a static MCP configuration a human wrote and becomes a runtime decision against live quality metrics. Figures come from the product’s own landing page — marketing, with no independent verification of the endpoint count or the measurement claims.

Sources added 2026-09-07

• Claude Code’s three orchestration levels
• Boris on writing loops
• treg

2026-09 Update — Harness Engineering as a Failure-Driven Ratchet

Agent Harness Engineering - Addy Osmani turns the harness-over-model thesis into an operating discipline:

1. Trace every permanent rule to a witnessed failure or hard constraint. Imagined failure modes do not earn scarce system-prompt space.
2. Choose the cheapest reliable intervention. Missing knowledge belongs in context or a scoped skill; dangerous actions belong behind deterministic hooks; flattering self-review calls for an independent evaluator.
3. Make success silent and failure verbose. Passing checks add nothing to context. Failed checks inject exact repair evidence back into the loop.
4. Name the behavior each harness component exists to produce. If the behavior is unclear, the component is probably dead weight.
5. Delete controls when their assumptions expire. Better models remove some failure modes while making harder tasks reachable; the harness moves rather than growing forever.

The actionable extension for persistent agents is a failure ledger: record recurring failures, promote them into scoped skills or enforcement only after evidence, verify that the intervention changes outcomes, then prune controls that no longer repay their context cost. This is how a harness compounds without becoming a junk drawer.

2026-09 Update — Software Factories as Deterministic Workflow Programs

From My Super Simple Software Factory - IndyDevDan:

• Typed Handoff Envelopes Turn Agent Context into Restartable Workflow State — validated phase envelopes convert temporary model context into inspectable, restartable state; provenance and replay require additional controls.
• Keep Agent Workflows In-Distribution with Boring Primitives — standard formats reduce the explanation and surprise costs of custom orchestration abstractions.
• Agents propose; code disposes. Put planning, implementation, and semantic review inside agent phases; keep routing, schema validation, mechanical checks, persistence, and advancement decisions in deterministic code.
• Design for repeated unattended runs. The factory becomes valuable when a workflow will run hundreds of times and failed gates route back automatically. One-off features rarely justify the ceremony.

The constraints matter as much as the pattern. Shared session directories need isolated runs, append-only phase ownership, and scoped permissions. Rigid schemas belong at inter-phase boundaries, not around every tool action. Observability supports diagnosis but does not remove human monitoring until recovery paths prove reliable. Every permanent phase should keep earning its latency and token cost as models improve.

2026-09 Update — Hermes Runtime and Learning Mechanics

Tony Simons’s four-part Hermes Master Class connects the harness thesis to concrete runtime behavior:

• Hermes Agent Master Class Part 1 - How Hermes Agent Actually Processes Work — agent loop, layered prompts, registry dispatch, compression, and iteration budgets
• Hermes Agent Master Class Part 2 - The Choices That Compound — deployment, persistent state, and real-tool smoke tests
• Hermes Agent Master Class Part 3 - The Learning System — bounded memory, skills, background review, and curation
• Hermes Agent Master Class Part 4 - Skills as Executable SOPs — routing descriptions, procedure anatomy, progressive disclosure, and lifecycle maintenance
• Agent Loops Turn Tool Results into Continued Reasoning — tool output becomes evidence for the next model decision
• Stable Prompts Trade Immediate Memory Visibility for Cache Efficiency — durable memory writes and current-prompt visibility are distinct guarantees
• Agent Learning Is Curated Context, Not Model Training — context accumulation, not weight updates, drives practical specialization
• Procedural Memory Needs Lifecycle Management — executable SOPs need tests, consolidation, and reversible retirement

2026-09 Update — Hermes Tools, Scheduling, and Channels

Parts 5–7 of Tony Simons’s Hermes Master Class extend the runtime map from the inner loop to the agent’s operating surface:

• Hermes Agent Master Class Part 5 - Tools and Toolsets — registry discovery, capability bundles, dynamic availability, execution backends, and command gates
• Toolsets Turn Agent Access into a Surface-Specific Policy — tool bundles make capability exposure a per-interface policy decision
• Unavailable Tools Should Disappear from the Model Schema — prerequisite checks keep unusable tools outside the model’s action space
• Hermes Agent Master Class Part 6 - Cron Makes Hermes Infrastructure — isolated scheduled runs, skill injection, deterministic execution, delivery, and job chaining
• Deterministic Prechecks Should Decide Whether to Spend an Agent Turn — scripts should suppress model calls when state has not changed
• Scheduled Agent Prompts Are Restartable Specifications — unattended work needs explicit targets, inputs, procedures, and success conditions
• Hermes Agent Master Class Part 7 - Messaging Gateways Make Hermes Ambient — shared session routing, authorization, platform policy, and adapter fault isolation
• Agent Channels Are Capability and UX Policies — a channel governs permissions, notification cost, and presentation as well as transport
• Channel Failures Should Degrade Locally, Not Globally — circuit breakers contain a provider outage at its adapter boundary

2026-09 Update — Hermes Orchestration, Interaction, and Operations

Parts 8–12 complete Tony Simons’s system map:

• Hermes Agent Master Class Part 8 - Delegation and Subagents — isolated child context, self-contained briefs, least-privilege toolsets, async control, and bounded orchestration
• Hermes Agent Master Class Part 9 - Browser and Computer Use — accessibility-tree browser control, native desktop operation, hybrid routing, and visual-context compression
• Choose the Narrowest Interaction Surface That Can Complete the Task — use the least-authoritative surface capable of finishing the work
• Accessibility Trees Are the Shared Control Plane for Visual Agents — structured element identity provides a low-token control layer across web and desktop
• Hermes Agent Master Class Part 10 - Kanban as a Coordination Model — durable ownership, dependencies, workflow state, and lifecycle notes across profiles
• In-Flight Work Is a Distinct Memory Class — unfinished work belongs outside factual memory and procedural skills
• WIP Limits Turn Agent Queues into Backpressure — lane limits stop fast producers from burying constrained reviewers
• Hermes Agent Master Class Part 11 - The Admin Layer — profiles, dashboard, API, and the boundary between agent state and host execution
• Runtime Isolation Does Not Imply Tool-Execution Isolation — separate profiles can still share files, identities, processes, and credentials
• Multi-Agent Administration Separates Runtime Control and Integration Boundaries — profiles hold state, dashboards administer it, and APIs expose it
• Hermes Agent Master Class Part 12 - What to Skip What Breaks and How to Stay Sane — context limits, silent capability loss, profile footguns, curation, staged complexity, and mechanism selection

2026-09 Update — Minimal Loop, Composable Middleware

From How to Build a Custom Agent Harness, LangChain’s create_agent makes the harness architecture concrete:

• The base is a deliberately small model–tool loop configured with a model, tools, and a system prompt.
• Middleware attaches at startup, teardown, model-call, and tool-call boundaries. Each unit can own deterministic logic, tool lifecycle, cross-hook state, or stream routing.
• Task–harness fit is the selection rule: context needs, likely failures, policy sensitivity, environment, duration, and action risk determine which controls belong in the stack.
• This is a composition mechanism, not an isolation primitive. The harness can govern filesystem or shell access while the runtime remains responsible for compute, process, network, credential, and isolation guarantees.

The article adds no benchmark evidence for middleware or create_agent; “production-ready” and “works for most cases” remain vendor claims. Its practical contribution is the capability-to-middleware map, which complements the failure-driven component test in Agent Harness Engineering - Addy Osmani.

Related Notes

• Agentic Software Factories - Operating Model and Evidence September 2026 — evidence synthesis across production systems, long-horizon benchmarks, security controls, and a revised Home One build sequence
• Simon Willison - Agentic Engineering Patterns — guide-level synthesis of how harness capabilities become professional coding practices
• Git History Is an Authored Context Store for Coding Agents — commit history as durable, curated context for future agent sessions
• Git Makes Agent Experiments Recoverable — Git recovery and bisect as harness-accessible safety mechanisms, bounded by external disclosure
• Codebase Knowledge Graph Tools - Research and Recommendation — separates structural code graphs, hybrid retrieval, live symbol services, and Home One’s curated knowledge graph; recommends repository-local indexing only after measured navigation pain
• Harness as a Service - Infrastructure for the Agent Era — HaaS: the commercialization of harnesses as a rentable infrastructure category (Cursor SDK, Managed Agents, Foundry)
• Most PMs Cant Touch Production - Gus Trigos via Aman Khan — Runtime (YC): sandboxed agent sessions so PMs ship PRs directly, engineering becomes quality gate
• Agent Harness Landscape - April 2026 Update — post-leak landscape (Claw Code, Channels, OpenClaw cutoff)
• Agent Harness Comparison - Hermes vs Paperclip vs Claude Code Router — detailed feature comparison
• Dex Horthy - Context Engineering for AI Coding Workflows — context hygiene, dumb zone, Research → Plan → Implement
• Carl Vellotti - Claude Code Mastery Context Engineering — sub-agents for context isolation, builder-validator pattern
• Boris Cherny - Hidden Claude Code Features Thread — 15 features from Claude Code’s lead dev
• 5 Claude Code Patterns - Sequential to Headless — 5 escalating usage patterns
• Anthropic - Claude Managed Agents Launch — cloud-hosted meta-harness (brain/hands architecture)
• The Complete Guide to Building Skills for Claude - Anthropic — official skill architecture
• Spec-Driven Development and AI-Native SDLC - 2026 Analysis — specs as the planning layer
• Karpathy - LLM Wiki Pattern for Personal Knowledge Bases — the knowledge layer above the harness
• Karpathy - No Priors Code Agents Autoresearch — auto-research, claws, program.md pattern
• GAN-Inspired Agent Architecture - Generator Evaluator Loops — multi-agent generator/evaluator pattern inspired by GANs, from Anthropic’s long-running harness work
• Criteria-Based Grading for Subjective Agent Output — making subjective quality (design, UX) gradable through decomposed criteria with explicit weightings
• Sprint Contracts - Negotiated Agent Agreements — generator and evaluator negotiate “done” definition before each sprint
• Harness Simplification as Models Improve — the discipline of stripping scaffolding as model capabilities increase
• The 90 to 100 Percent Automation Gap — 95% automation requires 100% monitoring; only 100% lets the human walk away
• Cat Wu - Head of Product Claude Code Cowork at Anthropic — insider context on Claude Code leak (human error), OpenClaw cutoff (capacity), anti-pattern of over-customizing MCPs
• LLM Document Degradation — agentic tool use does not mitigate document corruption in delegated workflows (DELEGATE-52)
• Delegation Readiness and the Jagged Frontier — domain-specific trust calibration for AI delegation
• Agent Runtime - The Execution Layer Nobody Is Building — the runtime hosts the harness; compute substrate, sandbox, state model, lifecycle controller
• Runtime Shift - A New Form of Distributional Shift for Agents — agents learn the runtime during training; moving runtimes causes silent quality regressions
• Sandbox Isolation Primitives for Agent Systems — five isolation primitives (containers, Firecracker, gVisor, Kata, V8) with trade-offs for agent workloads
• Humanising LLM Outputs is Actually Dumb — challenges the “return one sentence” default: sub-agents should exchange structured state, with prose rendered only at the human boundary
• Your Agent Is Not the Model — the four-layer vocabulary (model / inference service / harness / agent system) and a symptom→layer triage table
• Local Inference Stack for Personal Agents — the serving layer beneath a harness — persistent KV cache is what makes local models usable for agent workloads
• The Solo Operator Playbook - Five Primitives — the five capabilities a harness has to cover before one person can run an operation on it
• Three Repos That Fix Claude Code — the argument that the model is adequate and the harness is where the gaps are