---
title: "Does AGENTS.md Help Coding Agents? What 2026 Research Actually Shows"
date: "2026-11-06"
updatedAt: "2026-11-10"
description: "Three 2026 studies disagree on whether AGENTS.md helps coding agents. What the evidence supports, what to cut, and a one-afternoon eval to test your file."
author: "am2rican5"
tags: ["AI", "Developer Tools", "Software Engineering"]
isActive: true
readingTime: "11 min read"
translationKey: "does-agents-md-help-coding-agents"
slug: "does-agents-md-help-coding-agents"
authorImage: "https://github.com/am2rican5.png"
---

## The File Everyone Writes and Nobody Tests

Open a few active repositories these days and you'll find one. Sometimes it's called `AGENTS.md`, sometimes `CLAUDE.md`, sometimes it lives in `.cursor/rules` or `.github/copilot-instructions.md`. Often it's all four. The usual origin story is the same: someone ran an `/init` command, the agent wrote 200 lines describing the repository back to itself, and the file has been sitting there ever since.

Then in February 2026 a paper landed with an uncomfortable headline result: across several agents and models, repository-level context files **did not generally improve task success, and increased inference cost by over 20%**. A month earlier, a different study had found the opposite kind of result: agents with an `AGENTS.md` **finished faster and used fewer output tokens**.

Both studies are right, because they measured different things. Once you see what each one measured, it's much clearer what belongs in the file and what doesn't.

## What the Studies Actually Measured

Here's the evidence as of November 2026, along with what each study does *not* tell you.

| Study | Setup | Main finding | What it doesn't show |
|---|---|---|---|
| [Gloaguen et al., *Evaluating AGENTS.md*](https://arxiv.org/abs/2602.11988) (Feb 2026) | SWE-bench Lite tasks with LLM-generated files, plus a new benchmark of issues from repos with developer-committed files; multiple agents and models | No significant improvement in task success; more steps and over 20% higher cost. Developer-written files beat LLM-generated ones. Repository overviews didn't help. Instructions *were* followed. | That a well-curated, task-specific file is useless |
| [Lulla et al., *On the Impact of AGENTS.md Files on the Efficiency of AI Coding Agents*](https://arxiv.org/abs/2601.20404) (Jan 2026) | 10 repositories, 124 pull-request tasks, with vs. without the file | 28.64% lower median runtime, 16.58% fewer median output tokens, comparable completion behavior | Anything about patch correctness |
| [*Do Context Files Help Coding Agents?*](https://aclanthology.org/2026.realm-1.5.pdf) (REALM 2026) | Claude Code and Codex, 17 real tasks across 3 repos, 288 evaluated runs | Context strategy didn't measurably move correctness. Failures came from implementation skill — feature design, pattern choice, wiring — not missing repo knowledge. | Small sample; the authors say so explicitly |
| [Shanto et al., *Toward an Evidence-Based Template for Agent Instruction Files*](https://arxiv.org/abs/2611.02383) (Nov 2026) | 3,387 instruction files from 1,000 GitHub repos across AGENTS.md, CLAUDE.md, Copilot instructions, and Cursor rules | 42.2% of instruction-bearing repos in the higher-popularity cohort maintain multiple formats; 16.1% of compared section pairs give divergent guidance; 40.4% of files were modified only once | Whether those files help at all |

Nobody found that context files make agents *smarter*. What they found is that agents **do what the file says**, which costs you when the file says the wrong things and pays off when it says the right ones.

## The Reconciliation: It's a Decision-Removal File, Not a Capability Upgrade

An agent's success on a hard task is mostly bounded by the model's ability to design and wire the change. A Markdown file in the repo root doesn't change that. The REALM ablation puts it bluntly: the real `AGENTS.md` never turned a near-miss into a pass.

What the file *can* change is the path the agent takes to get there. Every instruction is a decision the agent no longer has to make by exploring:

- Which build command is the real one?
- Which directory is generated and should never be edited by hand?
- Which tests are enough for this kind of change?

When the instruction removes a decision the agent would have gotten wrong or spent ten tool calls figuring out, you see the Lulla et al. result: shorter runs, fewer tokens. When the instruction adds work — "here's an overview of our architecture," "always run the full test suite," "consider performance" — the agent dutifully reads, explores, and tests more, and you see the Gloaguen et al. result: more steps, higher cost, same success rate.

The [Agentic AI Foundation's guide](https://aaif.io/blog/writing-an-effective-agents-md) boils this down to one question I now apply to every line:

> **Would knowing this change what the agent does?**

If not, the line is costing you tokens and diluting the lines that matter.

## What Earns a Line

In practice, the lines that pass that test fall into five buckets.

**1. Commands that differ from the obvious default.** If `npm test` works and is the right thing to run, the agent will find it. If the real command is `make build` because the Makefile injects linker flags, or `pnpm run generate` because a post-build script does the actual work, write it down. The AAIF guide cites Kubo telling agents to use `make build` rather than `go build`. An experienced Go developer would make the reasonable choice and still be wrong for that repo.

**2. Generated files and their source of truth.** "Don't edit generated code" is useless unless the agent can identify generated code. Name the path, and name what to edit instead:

```markdown
- `src/routeTree.gen.ts` is generated by the TanStack Router plugin.
  Never edit it. Add or rename files under `src/routes/` and let the dev server regenerate it.
```

**3. Boundaries that aren't visible in the code.** Dependency direction between packages, public API surfaces, migrations that need a human. "Follow the existing architecture" isn't an instruction. "`packages/core` must not import from `packages/ui`" is.

**4. A definition of done that scales with the change.** Not "run the tests," but which tests, from where, for which kind of change — and when the expensive suite is actually required.

**5. Traps that look reasonable but break things.** The places where a competent engineer new to the repo would make a sensible move and cause a regression. This is the highest-value category and the one `/init` can never generate, because the trap is precisely the thing that isn't obvious from reading the code.

## What to Cut

The research gives you permission to delete a lot:

- **Repository overviews and directory listings.** Gloaguen et al. found overviews didn't help agents reach relevant code faster. The agent can run `ls` and read `package.json`.
- **Generic engineering advice.** "Write clean code," "add tests," "handle errors." None of it tells the agent what *this* repo needs.
- **Style rules a formatter already enforces.** If Biome or Prettier enforces it, the config file is the source of truth. Prose is a weaker, second copy.
- **Duplicated docs.** Point to the contributing guide instead of pasting it. Two copies will drift, and the Shanto et al. data says they already do in about one in six section pairs.
- **Anything secret.** Credentials and private operational details never belong in a file that gets committed and read by tools.

## A Worked Example: This Blog's Repository

Here's the file I'm adding to the repository that serves this blog. It's a React 19 + TanStack Start + Vite site, prerendered and deployed to Cloudflare. Here's what an agent would plausibly get wrong in it:

- There are **two lockfiles** (`package-lock.json` and `pnpm-lock.yaml`). CI installs with `pnpm install --frozen-lockfile`. An agent that runs `npm install` "to add a package" updates the wrong lockfile, and CI breaks on the next push.
- Post frontmatter is read twice: once by the Markdown plugin, and once by a build-time SEO plugin that parses it with **regular expressions**. That parser only understands double-quoted single-line strings and **single-line** tag arrays. A perfectly valid multi-line YAML list builds without an error, and the post silently loses its tags in the generated `llms.txt`.
- The page template already renders the post title as the `<h1>`. A post body that starts with `# Title` produces two `<h1>`s.
- `vite build` alone doesn't produce the deployable site. `pnpm run generate` does the build, then prerenders every route and copies the feed, sitemap, and Markdown mirrors into the deploy directory.

None of these are visible from the file tree. All of them are the kind of mistake that passes a quick review. So the file is short, and every line is a trap:

```markdown
# AGENTS.md

## Commands
- Package manager is pnpm. Never run `npm install`; it rewrites `package-lock.json`, not `pnpm-lock.yaml`.
- Dev: `pnpm dev`. Lint/format: `pnpm lint` (Biome, writes fixes).
- Deployable build: `pnpm run generate` (vite build + prerender into `dist/static`).
  `vite build` alone is not a release build.

## Generated files
- `src/routeTree.gen.ts` is generated by TanStack Router. Never edit by hand.

## Posts (`src/content/posts/{en,ko}/<slug>.md`)
- The filename is the slug. Korean translations share `translationKey` with the English post.
- Frontmatter values must be double-quoted on one line. `tags` must be a one-line array:
  `tags: ["AI", "Developer Tools"]`. `src/plugins/seo-generate.ts` parses frontmatter with regex.
- Do not start the body with `#`. The page renders the title as the only `<h1>`; start at `##`.
- Set `isActive: false` for drafts. Inactive posts are excluded from routes, feed, sitemap, and llms.txt.

## Done means
- Content-only change: `pnpm run generate` succeeds and the post appears in `dist/static/feed.xml`.
- Code change: `pnpm lint` and `pnpm run generate` both pass.
```

That's around twenty lines. Compare it with what a typical `/init` pass produces: a tour of the folder structure, a list of the dependencies in `package.json`, and a paragraph telling the agent the project "uses modern React best practices." Every one of those lines is discoverable. None of them would have prevented any of the four mistakes above.

## Enforce, Don't Plead

Look at the frontmatter rule again. It's in the file because nobody, agent or human, reliably remembers a rule like that. But a rule that matters this much shouldn't live only in prose.

This is the part of [harness engineering](https://martinfowler.com/articles/harness-engineering.html) that context files can't replace. If a rule can be checked mechanically, check it mechanically, and let `AGENTS.md` tell the agent the check exists:

```javascript
// scripts/check-frontmatter.mjs — fails CI if a post would break the SEO plugin
import fs from "node:fs";
import path from "node:path";

const root = "src/content/posts";
let failed = false;

for (const locale of ["en", "ko"]) {
  const dir = path.join(root, locale);
  for (const file of fs.readdirSync(dir).filter((f) => f.endsWith(".md"))) {
    const src = fs.readFileSync(path.join(dir, file), "utf8");
    const fm = src.match(/^---\r?\n([\s\S]*?)\r?\n---/)?.[1];
    const errors = [];
    if (!fm) errors.push("missing frontmatter");
    else {
      for (const key of ["title", "date", "description"]) {
        if (!new RegExp("^" + key + ':\\s*".+"\\s*$', "m").test(fm)) errors.push(key + " must be a quoted one-line string");
      }
      if (!/^tags:\s*\[.*\]\s*$/m.test(fm)) errors.push("tags must be a one-line array");
      // Ignore fenced code blocks: a "# comment" in a bash snippet is not a heading.
      const body = src.slice(src.indexOf("---", 3) + 3).replace(/```[\s\S]*?```/g, "");
      if (/^#\s/m.test(body)) errors.push("body must not contain an h1 (# ...)");
    }
    if (errors.length) {
      failed = true;
      console.error(locale + "/" + file + ": " + errors.join("; "));
    }
  }
}
process.exit(failed ? 1 : 0);
```

Wire it into CI before the build step, and the instruction in `AGENTS.md` becomes a hint that saves the agent a failed run rather than the only thing standing between you and a broken feed. Instructions shape behavior; checks guarantee it.

## Test Your File in One Afternoon

The common thread across every study above is that the effect of a context file is **specific to the repo, the agent, and the task mix**. A benchmark can't tell you whether *your* file helps. A small experiment can.

1. **Pick 5–8 tasks you already know the answer to.** Recently merged PRs are ideal: you have the final diff and, ideally, a test that proves it works. Mix sizes — a one-file fix, a cross-cutting change, a content or config change.
2. **Reset to the parent commit for each task** and give the agent the original issue text, nothing more.
3. **Run each task at least three times per condition**: no file, your current file, your trimmed file. Agents are noisy; a single run tells you almost nothing. In AAIF's small [five-run experiment](https://aaif.io/blog/writing-an-effective-agents-md) (one repo, one agent), two of five runs without a file wandered off to reorient or ran an unrequested production build on the harder task; none with the file did. One run per condition would have missed that entirely.
4. **Record four numbers per run**: did the tests pass; how many wrong commands did the agent run (wrong package manager, wrong build, edits to generated files); how many tool calls or tokens it used; and how many review comments you'd need to leave.
5. **Keep the lines that move a number. Delete the rest.**

Five tasks, three conditions, three runs is 45 runs. With most agents that's an afternoon of mostly waiting. It's also the only evidence about your file that actually applies to your file.

## One Source of Truth Across Tools

The Shanto et al. finding that 42% of popular repos with instruction files maintain several formats — and that one in six comparable sections disagree — is the most avoidable problem on this list.

Pick one file as canonical. Make the others thin pointers to it, or symlinks if your tools follow them (check — tool behavior here varies and changes). If a tool needs genuinely tool-specific settings, keep only those in its own file. The [AGENTS.md convention](https://agents.md/) also supports nested files: a file closer to the edited code takes precedence, which is the right place for rules that only apply to `services/api` or `packages/ui`.

## Treat It Like Code

The 40% figure — files edited once and never again — is the real failure mode. A context file that was accurate when someone ran `/init` and has drifted since is worse than no file, because the agent *will* follow it.

So treat it the way you'd treat any other config that changes runtime behavior:

- **Review changes to it** like code. A one-line edit changes what every agent run does.
- **Update it in the same PR** that changes a build command, moves a generated directory, or adds a boundary.
- **Add a line when the same mistake shows up twice** in review. Remove a line when CI starts enforcing it.
- **Re-run your afternoon eval** after model upgrades. Assumptions about what the agent can't figure out on its own expire.

## The Checklist

Run this against your file:

- Every line passes "would this change what the agent does?"
- Commands are exact, and only listed where they differ from the obvious default
- Generated paths are named, with the source to edit instead
- Boundaries are stated as concrete rules, not "follow the architecture"
- "Done" scales with the size and kind of change
- No overviews, directory listings, generic advice, or secrets
- Mechanically checkable rules are also enforced by CI or hooks
- One canonical file; other formats point to it
- Someone owns it, and it changes in the same PR as the thing it describes
- You've measured it on your own tasks at least once

`AGENTS.md` is a short list of things the agent would otherwise get wrong. The shorter and truer that list is, the better it works.
