Agent Docs

Agent Docs

Write documentation that AI agents can efficiently consume. Based on Vercel benchmarks and industry standards (AGENTS.md, llms.txt, CLAUDE.md).

The Hybrid Context Hierarchy

Three-layer architecture for optimal agent performance:

Layer 1: Constitution (Inline)

• Always in context. 2,000–4,000 tokens max.
• # AGENTS.md
• > Context: Next.js 16 | Tailwind | Supabase
• ## 🚨 CRITICAL
• NO SECRETS in output
• Use `app/` directory ONLY
• ## 📚 DOCS INDEX (use read_file)
• Auth: `docs/auth/llms.txt`
• DB: `docs/db/schema.md`
• Include:
• Security rules, architecture constraints
• Build/test/lint commands (top for primacy bias)
• Documentation map (where to find more)

Layer 2: Reference Library (Local Retrieval)

Fetched on demand. 1K–5K token chunks. Framework-specific guides Detailed style guides API schemas

Layer 3: Research Assistant (External)

Gated by allow-lists. Edge cases only. Latest library updates Stack Overflow for obscure errors Third-party llms.txt

Why This Works

Vercel Benchmark (2026): ApproachPass RateTool-based retrieval53%Retrieval + prompting79%Inline AGENTS.md100% Root cause: Meta-cognitive failure. Agents don't know what they don't know—they assume training data is sufficient. Inline docs bypass this entirely.

1. Compressed Index > Full Docs

An 8KB compressed index outperforms a 40KB full dump. Compress to: File paths (where code lives) Function signatures (names + types only) Negative constraints ("Do NOT use X")

2. Structure for Chunking

RAG systems split at headers. Each section must be self-contained: ## Database Setup ← Chunk boundary Prerequisites: PostgreSQL 14+ 1. Create database... Rules: Front-load key info (chunkers truncate) Descriptive headers (agents search by header text)

3. Inline Over Links

Agents can't autonomously browse. Each link = tool call + latency + potential failure. ApproachToken LoadAgent SuccessFull inline~12K✅ HighLinks only~2K❌ Requires fetchingHybrid~4K base✅ Best of both

4. The "Lost in the Middle" Problem

LLMs have U-shaped attention: Strong: Start of context (primacy) Strong: End of context (recency) Weak: Middle of context Solution: Put critical rules at TOP of AGENTS.md. Governance first, details later.

5. Signal-to-Noise Ratio

Strip everything that isn't essential: No "Welcome to..." preambles No marketing text No changelogs in core docs Formats like llms.txt and AGENTS.md mechanically increase SNR.

llms.txt Standard

• Machine-readable doc index for agents:
• # Project Name
• > One-line project description.
• ## Authentication
• [Setup](docs/auth/setup.md): Environment vars and init
• [Server](docs/auth/server.md): Cookie handling
• ## Database
• [Schema](docs/db/schema.md): Full Prisma schema
• Location: /llms.txt at domain root
• Companion: /llms-full.txt — full concatenated docs, HTML stripped

Inline = Trusted

AGENTS.md is part of your codebase. Controlled, version-pinned.

External = Attack Surface

Indirect prompt injection via hidden text SSRF risks if agents can browse freely Dependency on external uptime Mitigation: Domain allow-lists, human-in-the-loop for external retrieval.

Anti-Patterns

Pasting 50 pages — triggers "Lost in the Middle" "See external docs" — agents can't browse autonomously Generic advice — "Write clean code" (use specific constraints) TOC-only docs — indexes without content Trusting retrieval alone — 53% vs 100% pass rate

Advanced Patterns

For detailed guidance on RAG optimization, multi-framework docs, and API templates, see references/advanced-patterns.md.

Validation Checklist

Critical governance at TOP of doc Total inline context under 4K tokens Each H2 section self-contained No external links without inline summary Negative constraints explicit ("Do NOT...") File paths and signatures, not full code