John Young

Agent Runtime

AGENTS.md vs CLAUDE.md: Swap the Agent, Keep the Harness

37 min read 53 sources

Abstract
  1. When your AI coding agent has a red-status day, or worse, a quiet week where it keeps answering but the answers get worse and nobody declares anything, the fix isn’t a second subscription: it’s a repo where swapping the agent underneath is a config flip, not a migration.
  2. Sort your harness into two piles: the instruction file (one AGENTS.md at the root, with a CLAUDE.md that just imports it) ports across vendors almost for free, though the evidence it improves task success is thin, while hooks, permissions, subagents, and MCP client config don’t share a schema and have to be rebuilt per vendor.
  3. Put the guarantees that actually must hold, pre-commit checks, GitHub rulesets with the admin bypass closed, plain git worktrees, outside any agent’s config entirely, since no vendor’s schema can reach them and a sandbox that silently degrades to unsandboxed is a convention, not a boundary.
  4. Vendor outages cluster within a provider but not across providers, so keeping a second vendor warm buys an uncorrelated failure schedule rather than guaranteed uptime, and because the same model scores differently depending on which harness runs it, the only way to know a swap will work is testing it on your own tasks before the incident, not during it.
The plain-language edition for readers outside AI engineering no background needed · 10 min

This is about the AI tools that write code alongside programmers, and how to set one up so that if the company behind it has a bad day, you can switch to a different company’s version in minutes instead of losing a week to rebuilding everything.

The big idea

Think of an AI coding tool as a temp worker supplied by a staffing agency. You keep a written procedures binder at your desk that any competent temp could pick up and follow. But you also have things that only work because of who is currently on shift: a badge that opens certain doors, a hookup to your internal phone system, a specific way you’ve trained this one worker to flag something risky. If the agency’s phones go down, or worse, if the worker they send starts doing sloppier work without anyone officially saying so, you want to call a different agency and have the new temp be productive fast. That only works if you already know which parts of your setup are in the shared binder and which parts live in that one worker’s badge and training. The post’s argument is that most people never sort this out, so a switch turns into a slow rebuild instead of a quick handoff. Some parts genuinely do carry over. Most don’t, and pretending otherwise is what costs you the sprint.

Plan for two kinds of bad day, not one

The obvious bad day is when the company’s systems are visibly down, the equivalent of the agency’s phone lines going dead. Everyone notices, and the company itself usually posts about it.

The costlier bad day is quieter: the worker is still showing up and answering, but the answers have gotten worse, and nobody has officially declared anything wrong. In one real case, a routing bug misdirected messages for about a month before the company even opened a public report about it, and its own internal checks missed the problem because, as the company put it, the AI often recovers well from isolated mistakes, which hid the pattern.

A cheaper first move than switching companies entirely is switching how you reach the same company, the way you might call a different branch office instead of a different agency. On the worst hour of one real incident, the company’s own numbers showed that requests routed through a backup path failed far less often than requests going through the main path, sometimes by a thousand times less. That fix is nearly free, but it only helps with plumbing problems. It does nothing if the worker itself has quietly gotten worse.

A caveat worth keeping: a backup company is not a clean safety net either. Two different companies can share the same underlying road or landlord, meaning both can go down together even though they are technically separate businesses. One real day in late 2025 saw one AI company’s site go down and, separately, a major infrastructure provider have its own outage in the same window, though neither one’s report named the other as the cause. That connection is the author’s own reasoning from two side-by-side incidents, not something either company confirmed. Research on outage data also found that two services from the same company fail together on the same day more than 80% of the time in one dataset, while services from different companies showed no such pattern there, but that same research warns this is one dataset, not a universal law, since a different company’s own app and its programming interface correlated much more than that elsewhere.

Sort your setup into what transfers and what doesn’t

Once you accept you might need to hand the work to a different company’s worker, the next step is figuring out which pieces of your setup are in the shared procedures binder and which pieces are wired to one specific worker.

Interestingly, one of the AI companies has already published its own version of this sorting exercise as part of a tool that imports another company’s setup. It maps six kinds of things, instruction files, settings, automated guardrails, shortcut commands, sub-workers, and outside-tool connections, onto its own format, and then hands you a checklist of things to double-check afterward because the mapping isn’t perfect. Research across thousands of real project setups backs the same split: most projects use only one company’s tool, a smaller share run two, and a notable share keep their instructions in the shared, cross-company format specifically so they aren’t locked in.

The shared instructions file is the one thing that really does transfer, but it’s not where the value is

Most of these AI coding tools can read the exact same plain-text instructions file, typically named AGENTS.md, sitting at the top of your project. The one holdout is Claude Code, which by default only reads a file named CLAUDE.md. The fix is simple: make that file one line long, pointing at the shared AGENTS.md file, so Claude Code effectively reads the same handbook as everyone else. One real-world proof of this working: when a company retired one of its own AI tools and moved everyone to a replacement, it explicitly said both tools used identical instruction-file rules and nobody had to change anything. That’s the rare case of a vendor swap actually holding up.

Here’s the caveat that has to survive: making the instructions file portable doesn’t mean the instructions file is doing much work. Research found that having one of these instruction files does not generally improve how often the AI succeeds at a task, and it makes each request more expensive to run. A carefully written file did slightly better than an AI-generated one, but the improvement was small enough that researchers couldn’t call it a real, reliable effect. So carry this file across companies because it’s nearly free to do, not because it’s the thing standing between you and a bad outcome.

The guardrails, permissions, and connections don’t transfer, and rebuilding them is real work

This is the part that actually costs an afternoon. Automated guardrails (rules like “block any attempt to commit a secret password”), permission settings (how much the AI is allowed to do without asking), sub-workers the AI can delegate to, and connections to outside tools are all built differently by each company. The intent behind a guardrail, like “don’t let this slip through,” is the same everywhere. The file format, the names of the trigger moments, and the way you say “stop, that’s not allowed” are all different per company, with only one small convention (a specific error code meaning “blocked”) surviving the trip in most cases.

There’s a partial exception worth keeping honest: the reusable “skill” files some of these tools use have a small shared core of fields that do work across companies. If you stick to only that shared core, the same skill file can work in several tools. Add extra fields specific to one company, though, and at least one tool will flatly refuse to load the file rather than just ignoring the extra part. Outside-tool connections work similarly: the actual outside service you’re connecting to carries over fine, but the file that tells each AI tool how to reach it has to be rewritten per company.

Put the rules that must always hold outside any AI tool’s reach

The most reliable fix for all of this is to move anything that absolutely must hold true out of the AI tool entirely, into places no AI company controls: a separate folder (a “worktree”) per task so a half-finished session from one tool doesn’t collide with another, and rules enforced by your code-hosting platform itself, like blocking anyone from skipping required checks before merging code. Even the AI company that sells the shared instructions file agrees you should pair it with outside enforcement, like automated tests and code checks that catch problems no matter which AI wrote the code.

One more honest caveat: putting your AI inside a sandboxed container is sometimes treated as a hard safety boundary, but one company says plainly that it isn’t one by default, it’s a convention, and processes running alongside the AI can still act unconstrained on the underlying machine. If the sandbox can’t start for some reason, at least one tool will quietly fall back to running unsandboxed rather than stopping and warning you loudly, unless you specifically configure it not to. A fence that quietly opens itself when it can’t lock isn’t really a fence, so real enforcement belongs in the places outside the AI tool, not inside a permission setting.

Test the substitute worker on your own tasks before you need them

The same underlying AI model can perform very differently depending on which company’s tool is wrapping it, the same way the same recipe can come out differently depending on which kitchen and equipment it’s cooked in. Published comparison scores back this up: identical AI models scored quite differently on the same benchmark depending on which tool ran them, and this wasn’t always in favor of the model’s home company’s own tool. In one of three cases measured, a rival tool actually beat the model’s own company’s version.

The author is upfront that pinning this difference specifically to “the tool matters, not just the model” is his own reasoning from the data, not a claim the benchmark’s publishers made themselves; they just published the numbers without arguing about the cause. Either way, the practical point holds: a published leaderboard gap doesn’t tell you which company’s assistant will actually do your specific job well. You have to run your own tasks through the candidate before an emergency forces the decision.

What this means for you

If you rely on an AI coding assistant, don’t wait for an outage to discover which parts of your setup are locked to one company. Put your instructions in the shared, plain-text format everyone can read, since it costs almost nothing to keep portable, even though it isn’t the thing doing the heavy lifting. Expect to spend real setup time translating your guardrails and permission rules for each company you might switch to, ahead of time, not during the emergency. And put anything that truly must hold, like “never let this get merged without tests passing,” into places no AI vendor controls at all. Then actually test the backup assistant on your own work before you need it, because a leaderboard number won’t tell you how it performs on your project.

The technical terms, in plain words

  • AGENTS.md = the shared, plain-text instructions file most AI coding tools can read, no matter which company made the tool.
  • CLAUDE.md = the instructions file name one specific tool, Claude Code, looks for by default; you can make it just point at AGENTS.md instead of duplicating it.
  • Coding agent / AI coding assistant = an AI tool that writes and edits code for you, following instructions and asking permission for certain actions.
  • Harness = the overall setup around the AI: its instructions, guardrails, permissions, and connections, as opposed to the underlying AI model itself.
  • Hooks = automated guardrail rules that fire at specific moments, like “before this file gets saved, check it for passwords.”
  • MCP / MCP server = a connection that lets the AI reach an outside tool or service, like a database or a search tool.
  • Permissions / permission modes = settings controlling how much the AI is allowed to do on its own before it has to ask you first.
  • Subagent = a smaller helper AI the main assistant can delegate part of a task to.
  • Worktree = a separate folder for one task, so different work-in-progress doesn’t get tangled together.
  • Ruleset / branch protection = rules set at the code-hosting platform level (not inside the AI tool) that block certain changes unless conditions like passing tests are met.
  • Sandbox / container = an isolated environment meant to contain what the AI can affect, though the post notes it’s often a convention rather than a guaranteed boundary.
  • Serving path = which server route your request to the AI travels through; switching this is cheaper than switching companies entirely.
  • Status page / incident = the company’s own public log of when its service was degraded or down.
  • Model regression = the AI quietly getting worse at its job without an outage being declared.
  • Skill / SKILL.md = a reusable, packaged set of instructions the AI can use for a specific kind of task.
  • Benchmark / leaderboard = a published test comparing how different AI setups score on the same tasks.

Keep reading: the full version, with the research and sources ↓

Roughly 30% of Claude Code users who made requests during Anthropic’s August 2025 routing bug had at least one message routed to the wrong server type. The company’s own evals never caught it (Anthropic: A Postmortem of Three Recent Issues1). If your team’s workflow only runs on one vendor’s agent, every incident like that is your outage. The fix is not a second subscription: it is a repo where the agent underneath is a config flip rather than a migration.

Treat the Vendor as Your Failure Domain

Plan the swap for two triggers, not one. The obvious trigger is the red status day. The one that actually costs you a sprint is the quiet week when the model is still answering, the answers are worse, and nobody has declared anything.

The red status day is easy to name because the vendor names it for you. On 2026-09-03 Anthropic logged “Elevated errors for multiple models” at impact major, 13:26 to 16:23 UTC, listing claude.ai, the Claude API, Claude Code and Claude Cowork as affected together (Anthropic Status: Elevated Errors for Multiple Models2). That pattern repeats. A single model incident on 2026-07-17 took the same four surfaces down for five and a half hours (Anthropic Status: Elevated Errors on Sonnet 5 and Haiku 4.53). On 2026-08-28 the cause was not the model at all: “an issue with an upstream cloud provider affecting Claude Cowork and Claude Code on the web” (Anthropic Status: Elevated Errors on Claude Code and Claude Cowork4).

The blast radius of a vendor incident is the vendor, not the model. Chu et al. measured this across public status-page data through 2024-08-31 and found that for Anthropic’s services, the likelihood of any two services experiencing outages on the same day is over 80%. Their cross-provider finding is the useful half:

“There is no correlation observed between services from different providers.”

Carry the qualifiers, because they are load-bearing. The intra-vendor correlation is Anthropic-specific in that dataset: OpenAI’s API-to-ChatGPT same-day co-occurrence is 49.21%. The paper hedges the mechanism to different cloud infrastructures, and says the lack of correlation suggests using one service as the other’s backup rather than proving it (Chu et al.5).

The second vendor is no fixed point either. OpenAI’s own write-up of a June 2025 incident reports ChatGPT error rates peaking near 35% and API error rates near 25%, with recovery slowed by “The absence of break-glass tooling to rapidly restore network connectivity on affected nodes” (OpenAI Status: Elevated Error Rates6). You are not buying reliability by keeping a second agent warm. You are buying an uncorrelated failure schedule.

The second trigger leaves no incident to point at. Anthropic’s routing bug started on August 5, 2025 and the public status entry did not open until September 9, crediting community reports for isolating it (Anthropic Status: Model Output Quality7). The postmortem is blunt about why internal monitoring missed it:

“The evaluations we ran simply didn’t capture the degradation users were reporting, in part because Claude often recovers well from isolated mistakes.”

Date-stamp any incident count you quote, because the status feed is a rolling window that ages entries out. As of 2026-09-07 the page shows 99.44% 90-day uptime for Claude Code and 99.5% for the Claude API (Anthropic: Claude Status8).

Author’s judgment. The shared edge is also a failure domain, which limits how much a second vendor buys you. This follows from two sourced premises and is stated by neither: OpenAI attributed its 2025-11-18 outage to “an issue with one of our third-party service providers” without naming one (OpenAI Status: Access Issues Affecting OpenAI Websites9), and Cloudflare’s postmortem for the same day traces its own core-traffic failure, 11:20 to 17:06 UTC, to a database permissions change that doubled a Bot Management feature file (Cloudflare: Cloudflare Outage on November 18, 202510). The windows overlap. Neither names the other.

Everything below traces one worked example, the September 3 drill: a team mid-task on a feature-shipping-rates branch under Claude Code when incident 461yvfrzpwtt opened, who moved to Codex CLI for the afternoon and came back the next morning. The trigger is real and has a permalink; the team is invented. Each section asks what the drill hit in that layer.

Run a Second Serving Path Before a Second Vendor

If the trigger you fear is an API incident rather than a model regression, the cheapest hedge is the same agent pointed somewhere else. Anthropic’s own numbers make the case. At the worst impacted hour on August 31, 16% of Sonnet 4 requests were affected on first-party infrastructure, while misrouted traffic peaked at 0.18% on Bedrock and stayed under 0.0004% on Vertex AI (Anthropic: A Postmortem of Three Recent Issues1). Same model, same bug, three orders of magnitude of difference by serving path.

The cheaper hedge: same agent, different serving path
# Route Claude Code through Amazon Bedrock
export CLAUDE_CODE_USE_BEDROCK=1
export AWS_REGION=us-east-1

# Or through a gateway your organization already runs
export ANTHROPIC_BASE_URL=https://llm-gateway.internal

Both flips cost you something, so price them before the drill day. On Bedrock the /logout command is unavailable because authentication runs through AWS credentials, and the WebSearch tool is not available at all. Unpinned aliases such as sonnet and opus resolve to a built-in default that can lag the newest release, and Claude Code falls back to an earlier or lower-tier model at startup when that default is unavailable (Claude Code Docs: Claude Code on Amazon Bedrock11). A gateway moves the maintenance burden onto you: “Claude Code adds capabilities with each release, and a gateway that doesn’t forward them breaks the corresponding features, so the gateway product needs to be kept updated as Claude Code evolves” (Claude Code Docs: Other LLM Gateways12). Neither flip hedges a model regression or a capability gap, which is why the rest of this post exists.

Sort the Harness Into Two Piles

Inventory first, then sort. If you have not enumerated what is actually in your harness, start with the harness audit; this post sorts that inventory rather than re-deriving it. The sort runs on one test: is this artifact read by name and format across vendors, or is it read by one vendor’s schema?

The useful surprise is that a vendor already published its own version of this sort. Codex’s import page maps six categories of another agent’s setup onto its own constructs, and then lists what to re-check afterward.

Imported itemDestination
Instruction filesAGENTS.md
settings.jsonconfig.toml
HooksCodex hooks
Slash commandsSkills
SubagentsCodex subagents
MCP server configurationCodex MCP configuration

The review list underneath is the sort’s answer key: permissions in imported skills and agents, MCP settings that use custom authentication, headers, environment variables or transports, hooks whose behavior may differ after import, plugins needing manual follow-up, and prompt templates that depend on arguments or file-path placeholders (Codex Docs: Import From Another Agent13). Read that as a map of the vendor-bound pile, written by a vendor with every incentive to make the move look painless.

The research points the same direction. Galster et al., across 2,853 repositories, recommend that “developers who rely on multiple tools should maintain an AGENTS.md file as the shared core configuration, given its cross-tool support and the reference patterns we observed,” with tool-specific files as adapters that reference that shared core (Galster et al.: Harness Engineering for Agentic AI Coding Tools14). The same study found 2,015 repos (70.6%) on a single tool, 295 (10.3%) configured for two, and 493 (17.3%) running AGENTS.md alone with no tool-specific artifact. A source-code study of eleven agent runtimes found skills leading MCP in adoption at 9 of 11 against 8 of 11, and ACP hosting rival harnesses “with OpenHands running Claude Code, Codex, or Gemini CLI as interchangeable backends” (Barbaste et al.: Harness Engineering15). Its reading of the first half of 2026 states the whole problem as a trend line: “behavioral policy migrates from prompt prose to configuration.” The layer that carries your rules is the layer that keeps moving into vendor schemas.

The instruction file is also the weakest layer in enforcement terms, which is worth knowing before you over-invest in it. Anthropic says so directly: Claude treats memory files “as context, not enforced configuration. To block an action regardless of what Claude decides, use a PreToolUse hook instead” (Claude Code Docs: How Claude Remembers Your Project16).

In the September 3 drill, the first ten minutes were an inventory check, not a migration. AGENTS.md, the skills under .agents/skills/, and the MCP server processes were read unchanged by the Codex side. Everything under .claude/ was inert.

Make AGENTS.md the One Instruction File

Keep exactly one AGENTS.md at the repo root, with nested files only where a monorepo needs them, and give Claude Code a CLAUDE.md whose first line is an import. Claude Code is the one holdout of the four, and it tells you the workaround itself:

“Claude Code reads CLAUDE.md, not AGENTS.md. If your repository already uses AGENTS.md for other coding agents, create a CLAUDE.md that imports it so both tools read the same instructions without duplicating them.”

What the four-vendor repo root actually holds
AGENTS.md               # the one instruction file, read natively by 3 of 4
CLAUDE.md               # one line: @AGENTS.md
.codex/config.toml      # only if you need fallback filenames
.gemini/settings.json   # context.fileName: ["AGENTS.md", "CONTEXT.md", "GEMINI.md"]
                        # Cursor needs nothing; it reads AGENTS.md at root

Prefer the import over the symlink. A symlink works, but on Windows it requires Administrator privileges or Developer Mode, so the docs point you back at @AGENTS.md (Claude Code Docs: How Claude Remembers Your Project16). The year-long feature request asking for native AGENTS.md support was closed by pointing at that same import, and commenters in the thread report a Claude Code update that blocked writes through symlinked files (GitHub: Feature Request: Support AGENTS.md17). /import and /init are also the wrong tool for this: /import appends a one-time copy of instruction files rather than a live link (Claude Code Docs: How Claude Remembers Your Project16).

The other three flip with config, not files. Codex builds its instruction chain at startup, checking each directory in the order AGENTS.override.md, AGENTS.md, TEAM_GUIDE.md, .agents.md, with other names configurable through project_doc_fallback_filenames under a 32 KiB cap (Codex Docs: Custom Instructions With AGENTS.md18). Gemini CLI takes a context.fileName setting whose documented example is already ["AGENTS.md", "CONTEXT.md", "GEMINI.md"] (Gemini CLI Docs: Provide Context With GEMINI.md Files19). Cursor needs no flip at all: it reads AGENTS.md in the project root and subdirectories as “a plain markdown file without metadata or complex configurations,” unlike its own .mdc project rules (Cursor Docs: Rules20). Precedence is consistent enough to reason about: the closest AGENTS.md to the edited file wins, and explicit chat prompts override everything (AGENTS.md21).

This is the one layer with a real vendor-swap track record. Google retired consumer Gemini CLI access on June 18, 2026 and moved those users to Antigravity CLI (Google Developers Blog: Transitioning Gemini CLI to Antigravity CLI22). The migration guide’s line on context files is the whole argument for this section:

“Both CLI platforms utilize identical workspace context rules. No modifications are needed to your existing rule documents”

A vendor swapped its own tool out from under a user base and the instruction layer did not move. How to structure and budget what goes inside that file is a different problem, covered in the CLAUDE.md context hierarchy post; this section is about portability only.

The Most Portable Layer Is Not the Most Valuable

Port the instruction file because it is free, not because the swap’s risk lives there. The evidence that it earns its keep is thin:

“Surprisingly, we find that providing context files does not generally improve task success rates, while increasing inference cost by over 20% on average.”

Developer-written files did better than LLM-generated ones, but the gain was 2.4% on average and not statistically significant (Gloaguen et al.24). Vercel’s evals found the adjacent failure on the skills side: “In 56% of eval cases, the skill was never invoked. The agent had access to the documentation but didn’t use it,” while a compressed 8KB docs index embedded directly in AGENTS.md hit a 100% pass rate against 79% for skills (Vercel: AGENTS.md Outperforms Skills25). Treat the instruction file as cheap to move and light in effect. The rest of the harness is the opposite on both counts.

Rebuild Hooks and Permissions as Per-Tool Adapters

Never share this pile. Keep one adapter directory per vendor, treat each as generated output from a policy you own, and accept that the schemas do not meet at the field level. A hook that blocks a secret from reaching a commit is the same intent in all four tools and four different files in practice.

VendorHook configEvent vocabularyBlocking convention
Claude Code.claude/settings.json, plugin, skill and subagent frontmatterPreToolUse, PostToolUse, InstructionsLoaded, WorktreeCreate, TeammateIdle“Exit 2 means a blocking error.”
Codex CLI.codex/hooks.json or .codex/config.tomlPreToolUse, PermissionRequest, SubagentStop, Interruptexit code 2 plus the reason on stderr
Gemini CLIsettings.jsonBeforeTool, AfterTool, BeforeModel, BeforeToolSelection, PreCompress (11 events)per-hook semantics, different per event
Cursor.cursor/hooks.jsonbeforeShellExecution, afterFileEdit, preToolUse (21 events)exit 2, “This matches Claude Code behavior for compatibility”

Sources for the table, row by row: Claude Code Docs: Hooks Reference26, Codex Docs: Hooks Reference27, Gemini CLI Docs: Hooks Reference28, Cursor Docs: Agent Hooks29. Cursor is the only vendor that states the compatibility intent in writing, and even that covers the exit code, not the event names.

Work the adapter as a checklist, once per vendor you intend to keep warm:

  • Re-map the hook events. Your PreToolUse guard becomes a BeforeTool rule on Gemini and a beforeShellExecution or preToolUse entry on Cursor. The exit-2 convention survives the trip; the event name does not.
  • Re-express the permission policy. Claude Code gives you six named modes (default, acceptEdits, plan, auto, dontAsk, bypassPermissions) plus a classifier, and refuses to honor auto or bypassPermissions from .claude/settings.json at all (Claude Code Docs: Choose a Permission Mode30). Codex crosses approval_policy = "untrusted" against sandbox_mode = "workspace-write" or "danger-full-access" (Codex Docs: Advanced Configuration31). Gemini runs a priority-ordered TOML policy engine under a plan < default < autoEdit < yolo hierarchy (Gemini CLI Docs: Policy Engine32). Cursor takes rule strings like Shell(rm) and Read(.env*) where “Deny rules take precedence over allow rules” (Cursor Docs: CLI Permissions33). Which tier a task belongs in is the subject of the permission tiering post; this section only insists the tiers get re-expressed per tool.
  • Translate the subagent fields, not the directory. Cursor reads .claude/agents/ and .codex/agents/ for compatibility (Cursor Docs: Subagents34), so the file location is partly interoperable and the schema is not. Claude Code carries permissionMode, hooks, mcpServers and isolation: worktree (Claude Code Docs: Create Custom Subagents35), Codex requires TOML files defining name, description, developer_instructions and warns “the format may evolve as authoring and sharing mature” (Codex Docs: Subagents36), Gemini uses markdown with max_turns defaulting to 30 and timeout_mins to 10 (Gemini CLI Docs: Subagents37), and Cursor adds readonly and is_background.
  • Strip non-spec frontmatter from skills before you move them. Only six fields (name, description, license, compatibility, metadata, allowed-tools) are in the Agent Skills spec, and allowed-tools is flagged experimental with support that “may vary between agent implementations” (Agent Skills: Specification38). Anthropic’s own packaging path hard-fails on extras rather than ignoring them, with the literal error Unexpected key(s) in SKILL.md frontmatter (Claude Code Docs: Extend Claude With Skills39). Written to the spec surface, the same SKILL.md “works in Claude Code, Cursor (with rules), Gemini CLI, Codex, and any other harness that accepts system-prompt content” (Addy Osmani: Agent Skills40).
  • Re-emit the MCP client file per tool. The server process and the mcpServers shape port; the file does not. Claude Code reads .mcp.json and warns that “A JSON entry that has a url but no type is a configuration error, because Claude Code reads an entry with no type as a stdio server” (Claude Code Docs: MCP41). Codex uses [mcp_servers.<server-name>] in config.toml, scoped to trusted projects only (Codex Docs: Model Context Protocol42). Gemini keeps mcpServers in settings.json with a trust option that “bypasses all confirmation dialogs” (Gemini CLI Docs: MCP Servers43). Cursor wants .cursor/mcp.json with ${env:NAME} interpolation (Cursor Docs: MCP44).

That is a real afternoon of work, which is exactly why you do it before the incident and not during. The September 3 drill spent most of its rebuild time here: the pre-commit block on scripts/no-secrets.sh moved from a PreToolUse hook into .codex/hooks.json. The team’s acceptEdits posture became approval_policy = "on-request" paired with sandbox_mode = "workspace-write" before anyone let Codex run unattended.

Put Enforcement Where No Vendor Can Reach It

Move every rule that must hold out of the agent entirely. If a guarantee lives in a hook config, it is only as portable as that vendor’s schema, and it is off the moment someone runs a different tool. The vendor that sells you the instruction file agrees: “Pair AGENTS.md with infrastructure that enforces those rules: pre-commit hooks, linters, and type checkers catch issues before you see them” (Codex Docs: Customization45).

Three layers sit outside every vendor’s reach, and all three are boring on purpose.

The layer no agent config can turn off
# 1. A plain worktree per task. Gemini's own docs give this as the
#    manual equivalent of its .gemini/worktrees convention.
git worktree add ../shipping-rates -b feature-shipping-rates
cd ../shipping-rates && claude    # or: codex, gemini, cursor-agent

# 2. A ruleset on main, with the admin bypass closed and the
#    required check pinned to the app that sets it.
gh api repos/:owner/:repo/rulesets --method POST --input ruleset.json

The worktree line is the vendor’s own suggestion, not mine. Gemini CLI documents git worktree add as the manual path alongside its experimental .gemini/worktrees/ flag, and warns that it “does not automatically delete” your worktree or branch (Gemini CLI Docs: Git Worktrees46). Codex’s managed worktrees live in $CODEX_HOME/worktrees and keep the most recent 15, but the feature is scoped to the desktop app running parallel chats (Codex Docs: Worktrees47). A plain sibling directory is what both reduce to, and it costs nothing to standardize on.

Rulesets carry the guarantees. Multiple rulesets targeting the same branch are aggregated and “the most restrictive version of the rule applies.” Anyone with read access can view a repository’s active rulesets, which makes the policy auditable without admin (GitHub Docs: About Rulesets48). Two settings are the actual footgun. By default the restrictions “don’t apply to people with admin permissions to the repository or custom roles with the ‘bypass branch protections’ permission,” which you close with “Do not allow bypassing the above settings.” The second is the required status check: pin it to the app that sets it, because “If the status is set by any other person or integration, merging won’t be allowed” (GitHub Docs: About Protected Branches49).

The container is the third layer, and it is only a boundary if your organization makes it one:

“This is a convention rather than an enforcement boundary, because Claude Code does not require a container.”

The same page notes that under the built-in Bash sandbox, “MCP servers and hooks are separate processes that run unconstrained on the host,” and points at device management or software allowlisting as the enforcement mechanism (Claude Code Docs: Choose a Sandbox Environment50). The vendor sandbox is not liftable either. By default, if it cannot start because dependencies are missing or the platform is unsupported, Claude Code shows a warning and runs commands without sandboxing unless sandbox.failIfUnavailable is set to true (Claude Code Docs: Configure the Sandboxed Bash Tool51). A boundary that silently degrades to no boundary is a convention with better marketing. Unattended runs belong inside this layer rather than inside a permission mode, which is the argument in the loop engineering post.

This is the layer the drill did not have to think about. The ci/test check and the ruleset on main held identically on the afternoon’s pushes, whether Claude Code or Codex made them. Because feature-shipping-rates was its own worktree, the half-finished Claude Code session was still sitting there untouched the next morning.

Test the Swap on Your Own Tasks

Run your own task set under the second agent before the drill day, because the harness moves the number as much as the model does. Terminal-Bench 2.1 scores the same model weights at materially different accuracy depending on which harness runs them (Terminal-Bench 2.152).

ModelVendor’s own harnessTerminus 2Which harness wins
Opus 4.670.1% (Claude Code)63.8%vendor harness
GPT-5.477.3% (Codex CLI)54.8%vendor harness
Gemini 3.1 Pro67.1% (Gemini CLI)70.7%Terminus 2

The direction flips on the third row. The provider’s own harness is not automatically the best one for the provider’s own model.

Author’s judgment. That these gaps are substantially a harness effect is my inference, not a claim the benchmark makes. Terminal-Bench 2.1 publishes the agent-model table and argues nothing about cause; the premise it rests on is that the same model weights sit under both columns, so the difference has to come from the runtime around them.

A two-point leaderboard gap therefore tells you nothing about which agent will ship your work clean on the first try, and the tasks that expose the difference are yours. Pick the second agent the same way you would pick the first: on your own repo, with your own task specs, measured on your own definition of done. That argument is the whole subject of the coding agent leaderboard noise post. It is the one decision here that no amount of repo layout will make for you.

The Portability Ledger

One row per layer. The last column is what the September 3 drill actually hit. If you want the ledger filled in for your own repo rather than the drill’s, harness-portability-audit53 walks the root, sorts every artifact it finds into the two piles, and names the bridge that is missing: the @AGENTS.md line, the context.fileName key, the non-spec skill field, the .mcp.json entry with a url and no type.

LayerPorts across vendors?The flipWhat the drill hit
Vendor as failure domainNo flip existsPlan for two triggers; try a second serving path firstIncident 461yvfrzpwtt, 13:26 to 16:23 UTC
Instruction fileYes, by conventionCLAUDE.md holding @AGENTS.md, plus two config keysNothing; Codex read AGENTS.md unchanged
SkillsSpec surface onlyStrip frontmatter down to the six spec fieldsNothing; .agents/skills/ loaded as-is
MCPServer yes, client file noRe-emit .mcp.json as config.toml or settings.jsonRe-emitted, one file, ten minutes
HooksNoRe-map the event names; exit 2 survivesPreToolUse guard rebuilt in .codex/hooks.json
PermissionsNoRe-express the tier in each vendor’s grammaracceptEdits became approval_policy plus sandbox_mode
SubagentsLocation partly, fields noTranslate the frontmatter per schemaLeft behind for the afternoon
Worktrees, rulesets, containerYes; no vendor owns themNothing to flipHeld on every push, either agent
CapabilityNot portable, but measurableRun your own task set in advanceOnly revealed whether it was done

The pattern the ledger makes visible: the layer the keyword asks about is the layer that already works. AGENTS.md is solved, cheap, and light in effect. The swap is expensive in the adapter pile, and it is free in the enforcement pile only if you put the enforcement there first.

Sources

  1. Anthropic: A Postmortem of Three Recent Issuesanthropic.com/engineering/a-postmortem-of-three-recent-issues
  2. Anthropic Status: Elevated Errors for Multiple Modelsstatus.claude.com/incidents/461yvfrzpwtt
  3. Anthropic Status: Elevated Errors on Sonnet 5 and Haiku 4.5status.claude.com/incidents/7gpjd8n56rlq
  4. Anthropic Status: Elevated Errors on Claude Code and Claude Coworkstatus.claude.com/incidents/vr9tpk8w7zr8
  5. Chu et al.: An Empirical Characterization of Outages and Incidents in Public Services for LLMsarxiv.org/abs/2501.12469
  6. OpenAI Status: Elevated Error Ratesstatus.openai.com/incidents/01JXCAW3K3JAE0EP56AEZ7CBG3/write-up
  7. Anthropic Status: Model Output Qualitystatus.claude.com/incidents/72f99lh1cj2c
  8. Anthropic: Claude Statusstatus.claude.com
  9. OpenAI Status: Access Issues Affecting OpenAI Websitesstatus.openai.com/incidents/01KABE2437NJYKBFHT22SD3H92
  10. Cloudflare: Cloudflare Outage on November 18, 2025blog.cloudflare.com/18-november-2025-outage
  11. Claude Code Docs: Claude Code on Amazon Bedrockcode.claude.com/docs/en/amazon-bedrock
  12. Claude Code Docs: Other LLM Gatewayscode.claude.com/docs/en/llm-gateway
  13. Codex Docs: Import From Another Agentlearn.chatgpt.com/docs/import
  14. Galster et al.: Harness Engineering for Agentic AI Coding Toolsarxiv.org/abs/2602.14690
  15. Barbaste et al.: Harness Engineeringarxiv.org/abs/2609.00006
  16. Claude Code Docs: How Claude Remembers Your Projectcode.claude.com/docs/en/memory
  17. GitHub: Feature Request: Support AGENTS.mdgithub.com/anthropics/claude-code/issues/6235
  18. Codex Docs: Custom Instructions With AGENTS.mdlearn.chatgpt.com/docs/agent-configuration/agents-md
  19. Gemini CLI Docs: Provide Context With GEMINI.md Filesgeminicli.com/docs/cli/gemini-md
  20. Cursor Docs: Rulescursor.com/docs/rules
  21. AGENTS.mdagents.md
  22. Google Developers Blog: Transitioning Gemini CLI to Antigravity CLIdevelopers.googleblog.com/an-important-update-transitioning-gemini-cli-to-antigravity-cli
  23. Antigravity CLI Docs: Gemini CLI to Antigravity CLI Migration Guideantigravity.google/docs/cli/gcli-migration
  24. Gloaguen et al.: Evaluating AGENTS.mdarxiv.org/abs/2602.11988
  25. Vercel: AGENTS.md Outperforms Skillsvercel.com/blog/agents-md-outperforms-skills-in-our-agent-evals
  26. Claude Code Docs: Hooks Referencecode.claude.com/docs/en/hooks
  27. Codex Docs: Hooks Referencelearn.chatgpt.com/docs/hooks
  28. Gemini CLI Docs: Hooks Referencegeminicli.com/docs/hooks/reference
  29. Cursor Docs: Agent Hookscursor.com/docs/agent/hooks
  30. Claude Code Docs: Choose a Permission Modecode.claude.com/docs/en/permission-modes
  31. Codex Docs: Advanced Configurationlearn.chatgpt.com/docs/config-file/config-advanced
  32. Gemini CLI Docs: Policy Enginegeminicli.com/docs/reference/policy-engine
  33. Cursor Docs: CLI Permissionscursor.com/docs/cli/reference/permissions
  34. Cursor Docs: Subagentscursor.com/docs/agent/subagents
  35. Claude Code Docs: Create Custom Subagentscode.claude.com/docs/en/sub-agents
  36. Codex Docs: Subagentslearn.chatgpt.com/docs/agent-configuration/subagents
  37. Gemini CLI Docs: Subagentsgeminicli.com/docs/core/subagents
  38. Agent Skills: Specificationagentskills.io/specification
  39. Claude Code Docs: Extend Claude With Skillscode.claude.com/docs/en/skills
  40. Addy Osmani: Agent Skillsaddyosmani.com/blog/agent-skills
  41. Claude Code Docs: MCPcode.claude.com/docs/en/mcp
  42. Codex Docs: Model Context Protocollearn.chatgpt.com/docs/extend/mcp?surface=cli
  43. Gemini CLI Docs: MCP Serversgeminicli.com/docs/tools/mcp-server
  44. Cursor Docs: MCPcursor.com/docs/mcp
  45. Codex Docs: Customizationlearn.chatgpt.com/docs/customization/overview
  46. Gemini CLI Docs: Git Worktreesgeminicli.com/docs/cli/git-worktrees
  47. Codex Docs: Worktreeslearn.chatgpt.com/docs/environments/git-worktrees
  48. GitHub Docs: About Rulesetsdocs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/about-rulesets
  49. GitHub Docs: About Protected Branchesdocs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches
  50. Claude Code Docs: Choose a Sandbox Environmentcode.claude.com/docs/en/sandbox-environments
  51. Claude Code Docs: Configure the Sandboxed Bash Toolcode.claude.com/docs/en/sandboxing
  52. Terminal-Bench 2.1tbench.ai/news/terminal-bench-2-1
  53. agent-engineering-toolkit: harness-portability-auditgithub.com/johnayoung/agent-engineering-toolkit

← All essays