Monday 9 November 2026
A stale test command cost agents a third more calls
Five bullets
The one thing
When Instruction Files Go Stale measured what outdated AGENTS.md and CLAUDE.md files cost coding agents. With a stale test command, Claude Sonnet 5.5 needed about two extra tool calls per run. That is a third more than with the corrected command. For the other two stale instructions, the authors could not detect a difference.
The study took three real stale instructions from active open-source repos. One was a test command, one a code location, and one a procedure for adding a component. Agents fixed 22 maintenance tasks with each instruction left stale, corrected or removed. The authors analyzed 732 trajectories from two models, one tool call at a time.
Agents followed stale guidance when it was relevant to the task. Sometimes they followed it even when correct information existed elsewhere. They usually recovered, so the task still passed. None of them repaired the instruction file.
Correcting the file eliminated operations on obsolete paths. Removing the stale line prevented most of them. The skill that grows is keeping your instruction files true, because your agent will not.
Check the paths and commands in your instruction files, because your agent will not fix them.
Try this · 15 minutes
Find the stale lines in your agent's instruction file
- Open the AGENTS.md or CLAUDE.md in a repo you work in every day.
- List every test command, file path and procedure it tells the agent to follow.
- Run each command and check that each path exists in the current repo layout.
- Correct each stale line, or remove it if you cannot fix it now.
Stories
- TRYClaude Code's Don't Ask Again leaked passwords into public reposApiiro found passwords in 1,122 public GitHub files The secrets came from Claude Code's Don't Ask Again option. Approved commands that contain secrets get saved to permission settings files. Those files then get committed. Search your repos and .claude settings files for credentials, and keep local settings gitignored. It matters to anyone who approves commands that carry tokens or passwords.
- TRYStrong-to-cheap model handoffs cut agent API costs by 65.9%AgentBaton learned when to hand an agent task to a cheaper model Across 11 model pairs, it cut API costs by 65.9% on average while largely preserving accuracy. Handoffs from cheap to capable models generally underperformed. Separately, Hightouch reported a Jev model that asks Opus for help matched Opus accuracy at 4x lower cost. Worth testing if API spend on long agent tasks is your constraint.
- TRYHide agent-written PR descriptions behind a short human summarySimon Willison started collapsing agent-written PR descriptions He opens each PR with a short paragraph that carries his own judgment, sometimes with a screenshot. The full agent essay follows inside a <details><summary> block, which GitHub Markdown supports. He labels the summary with the model and effort level, such as Claude Opus 5.5 Medium. Useful for anyone whose reviewers have stopped reading agent PRs.
- TRYClaude Code v2.1.319 changes auto mode behind LLM gatewaysCalls the server cannot check are now asked about or denied. Set CLAUDE_CODE_AUTO_MODE_SERVER=0 to keep the built-in classifier.
- TRYopencode v2.2.6 starts scripted runs in a fresh git worktreeUse opencode run -w for isolated runs. Setting pool: true lets one MCP connection serve projects with the same setup.
- WATCHOpenAI Agents SDK v0.24 alpha keeps sessions in your own storageCustomStorage persists root history and reports uncertain tool calls for reconciliation. It is still alpha and changes executor write defaults.
- WATCHAgent memory formats are fragmenting, and one builder proposes a specLoreKit's author counts four incompatible memory shapes in a typical setup. For now, keep project context in files you control.
- WATCHMETR finds Anthropic's models improving about twice as fast as last yearThe 11-week study says Anthropic is using AI to build AI. Nothing to change today, but expect your tools to keep shifting.
Safe to ignore
57 sources read. 155 items. 14 shortlisted. 5 bullets.