Thursday 12 November 2026
26 developers distrusted agent-refactored tests. Tests as data help.
Five bullets
The one thing
Matteo Vaccari interviewed 26 Thoughtworks developers who build with coding agents. Asked if they trusted their tests, they said not much. The agent kept refactoring existing tests along with the code.
His answer, published on Martin Fowler's site, is to move the contract into test data. When agents write the code, developers become its customers. Customer tests become the main contract between human and AI. Kept as data, that contract is easier to protect from unexpected agent edits.
The approach borrows from older practice. It draws on acceptance tests from Extreme Programming, tabular tests from Go, and readable state from approval testing. Custom test runners and test DSLs used to be expensive to maintain. Now an agent can write them cheaply.
Agent-written tests often lean on mocks and confirm the current implementation, bugs included. A short table of inputs and expected outputs is something you can actually read. The skill that grows is writing the contract, not the test code.
Write one behavior as a table you own, and let the agent write the code that reads it.
Try this · 15 minutes
See whether your agent respects a test contract you wrote
- Pick one behavior your agent maintains. Write five inputs with their expected outputs.
- Save them as a plain table in a data file, separate from the test code.
- Ask your agent to write a small runner that reads the file and checks each row.
- On the next feature, check the diff to see whether the agent edited your table.
Stories
- TRYGemini Decision 1 answers bounded questions without generating output tokensGoogle launched the Gemini Decisions API with a new model, gemini-decision-1. It scores yes/no, multiple-choice and rubric questions in one pass, with a confidence score per answer. Input costs $0.10 per million tokens, and output is free. It suits engineers adding routing, tool filtering or policy checks to agent loops.
- TRYAWS ships an official local emulator, written by a swarm of agentsAWS released an official emulator for running tests against a local clone of its services. You do not need a paid API key. Agents wrote it in Rust, with 29,684 commits in the first three weeks. It currently needs Docker on macOS or Windows. Try it if your agents write code that calls AWS.
- TRYAtuin Threads resumes agent sessions in another harness on another machineAtuin added Threads in version 18.24 to search and sync agent sessions across machines. You can start a task in Claude Code, then resume it in Codex elsewhere. Sync is end-to-end encrypted, and capture is opt-in. It helps if you run agents on a laptop and a dev server.
- TRYllm 0.37.2 fixes two ways your OpenAI key could leakThe bugs affected OpenAI-compatible endpoints from other providers. Upgrade if you use extra-openai-models.yaml or llm openai endpoint add.
- WATCHPydantic AI 2.68 falls back to the next model at max_tokensRequestInput asks for approvals inside a run. It also adds a Gemini Decisions backend. Read the compatibility notes before upgrading.
- WATCHInfrathon: agent model rankings shift between short and 24-hour runsIn 135 of 150 runs, agents kept improving in the second half. Premature completion and unreliable testing held them back.
- WATCHMCP moves to two spec releases a yearReleases land on the last Tuesday of March and September. The next, version 2027-03-30, targets March 30, 2027.
- WATCHOpenAI showed voice steering and cloud handoff for CodexAWS also published a guide to running Codex with Amazon Bedrock as the model backend, starting with a least-privilege pilot.
Safe to ignore
57 sources read. 104 items. 13 shortlisted. 5 bullets.