Nobody Has It Figured Out: Notes from Martin Fowler’s Fragments

What the Thoughtworks Future of Software Development retreat suggests about testing, review, risk, and the work that follows cheap code generation.

The honest starting point

Martin Fowler has been publishing short reports from a Thoughtworks retreat on the future of software development. The participants spent several days comparing what they had actually learned from using AI in software teams. Across the reports, the most useful observation was that faster code generation moves pressure onto the rest of the delivery system.

Several participants arrived expecting the people in the room to have settled on a new method. They had not. Experienced teams were getting real value from the tools, but the practices around them were still provisional. That uncertainty is a better starting point than pretending the profession has already converged.

A team can now produce a plausible implementation before it has fully understood the requirement, the surrounding system, or the consequences of being wrong. That feels fast at the keyboard. It may not be fast by the time the software reaches production. The retreat notes are largely about that gap.

Code was only one part of the work

Most delivery delays do not come from the time required to type a function. Teams wait for decisions, chase unclear requirements, untangle dependencies, review changes, investigate failures, and coordinate releases. Generating the function more quickly helps, but it also exposes every weakness around it.

This explains why AI has such different effects across codebases. In a healthy system, an agent can make a change, run a meaningful test suite, and receive a useful signal. In a neglected system, it can produce more code without producing more confidence. The same tool accelerates whichever feedback loop already exists.

That is a more practical framing than asking whether AI makes an individual developer ten times faster. The interesting unit is the path from an idea to a verified change in production. Code generation occupies only part of that path.

Tests become instructions and evidence

The retreat repeatedly returns to test driven development because tests serve two roles when an agent writes the implementation. Before the change, a test can state the behavior in a form the agent can act on. After the change, the same test supplies evidence that the implementation meets that behavior.

This does not make every test a good specification. A weak test can reward the wrong behavior, encode an accidental implementation detail, or cover only the happy path. The engineer still has to decide what matters. The difference is that a precise executable example gives both the human and the agent a shared target.

The same principle extends beyond unit tests. Type checks, static analysis, contract tests, security policies, production assertions, and narrow benchmarks all turn a vague request into something that can be checked. As generation becomes cheaper, the quality of those checks matters more.

The middle of the loop becomes the job

One term from the retreat is the supervisory engineering middle loop. It describes the work between assigning a task and accepting the result. The engineer supplies context, examines the proposed change, notices where the agent misunderstood the system, and decides what should happen next.

Calling this prompt engineering makes it sound smaller than it is. Good supervision depends on understanding the architecture, the business rule, the failure modes, and the cost of an incorrect change. It also depends on knowing when a diff is easy to verify and when the task should be decomposed before any code is generated.

Review therefore stops being one undifferentiated activity. Some changes can be checked almost mechanically. Others need a domain expert, a security review, an experiment, or observation in production. Risk tiering is the sensible response. The amount and kind of evidence should depend on what the change can damage, not on how quickly it was produced.

The same change reaches developer tooling. The retreat uses the phrase agent experience for repositories, documentation, commands, and feedback that an agent can navigate reliably. A codebase organized for a person is not automatically legible to an agent. Clear boundaries, executable checks, useful error messages, and concise local instructions help both.

Cheap output can create expensive understanding

The retreat report uses the phrase cognitive debt for a problem that is already easy to recognize. A team accepts generated changes faster than its members can build a reliable mental model of them. The software may work, but fewer people understand why it works or how its parts interact.

Generated code is compatible with this principle, provided that the team’s understanding remains proportional to its responsibility. If a service is consequential enough that the team must repair it during an incident, someone needs a usable model of the service before that incident begins.

The same concern applies to apprenticeship. Junior work has historically combined useful output with repeated exposure to real systems. If agents absorb the routine output while senior engineers perform all verification, the organization may become more productive now while weakening the process that produces future senior engineers. Organizations can remove busywork while deliberately replacing the learning that once arrived as a byproduct of delivery.

Questions the retreat leaves open

The report is candid about the unresolved parts. Specialist roles may shrink as agents cross boundaries that once required separate expertise, or the specialists may become more valuable because they are the people able to verify difficult work. Token costs may become negligible, or heavy agent use may make them an operating constraint. Better specifications may strengthen iterative development, or they may tempt organizations back toward large batches handed to a machine.

Security is the most immediate concern. An agent can produce and execute changes more quickly than a traditional workflow, while prompt injection and excessive permissions create new paths to failure. Faster generation makes risk based controls more important, not less. The retreat does not offer a finished answer, but it correctly refuses to treat security as a cleanup step after the productivity gains arrive.

What I took from the retreat

Some old practices will change substantially. Making one activity inexpensive still leaves the surrounding system dependent on discipline.

Teams that benefit most from code generation will be able to state what they want, provide the right context, check the result at several levels, and recover when those checks miss something. That work is less visible than generating a large diff, but it is where speed becomes dependable delivery.

Sources