Some Simple Economics of AGI: Structured Notes
Catalini, Hui, and Wu examine what happens when AI makes work much cheaper to produce than to verify.
Most discussions of AI and work begin with capability. Can a model write the code, prepare the tax return, review the contract, or recommend a treatment? In Some Simple Economics of AGI, Christian Catalini, Xiang Hui, and Jane Wu begin somewhere else. They ask what it costs to verify the result.
That change in viewpoint is useful even if the strongest forms of artificial general intelligence never arrive. We already have systems that can produce more candidate work than people can examine carefully. Once production becomes cheap, confidence in the output becomes the scarce resource.
Production and verification do not improve together
Suppose an agent can draft one hundred contracts in the time a lawyer once needed for one. The organization has gained something only if it can determine which drafts are correct, whether important clauses are missing, and who is responsible for approving them. If each draft still demands an expert review, generation is no longer the limiting step.
The paper calls the difference between the falling cost of producing work and the stubborn cost of checking it the measurability gap. Some tasks have a narrow, objective result. A compiler can reject invalid code. A ledger can be reconciled. A shipment can be matched to a signed receipt. Other tasks depend on delayed outcomes, disputed values, incomplete evidence, or judgment that is difficult to encode.
This gives us a more useful way to discuss automation. A prestigious task is not protected merely because it involves expertise. A routine task is not automatically easy to automate. The practical question is whether the desired result can be specified and checked at a cost that still leaves an economic gain.
The paper maps tasks into four zones:
| Zone | Automation | Verification | Typical shape |
|---|---|---|---|
| Safe Industrial | Affordable | Affordable | Work with fast, objective feedback |
| Human Artisan | Expensive | Affordable | Work that remains easier to perform than automate |
| Pure Tacit | Expensive | Expensive | Work guided by experience where neither execution nor evaluation is easy to formalize |
| Runaway Risk | Affordable | Expensive | Output that can be generated at scale without a comparable ability to check it |
The last zone is the uncomfortable one. Deployment can look economically rational to each organization because generation is cheap, even while undetected errors accumulate across the larger system.
Cheap checks make good automation markets
The best candidates have feedback built into the work. Software can be compiled, tested, scanned, deployed to a small population, and observed. A payment can be balanced against an order and a bank record. A document extraction system can be evaluated against fields that a person already verified.
These checks are not perfect, but they make failure visible enough to improve the system. They also make it possible to route unusual cases to people instead of asking people to review everything.
By contrast, a strategy memo may look polished while resting on a bad assumption that will not become obvious for months. A clinical recommendation may require expertise, patient context, and evidence that is still developing. In those settings, plausible language can arrive much faster than reliable judgment.
Using AI to verify AI only moves the problem
An obvious response is to assign another model to review the first one. This is often useful. Independent sampling, adversarial review, and tool based checks can catch real errors. They do not eliminate the need for an external standard.
If the producer and reviewer share the same blind spot, agreement provides little evidence. A model grading an answer against an ambiguous prompt may reward the same plausible shortcut that produced the answer. Verification becomes stronger when it reaches outside the model: an executable test, a trusted record, a physical measurement, a human with relevant context, or an outcome observed later.
The distinction is between another opinion and a check anchored to the world. Both have value, but they are not interchangeable.
The labor problem is also a training problem
The paper also considers what happens when organizations remove junior work first. Routine assignments are economically attractive targets for automation, but they have also been how newcomers encounter real cases, make small mistakes, and learn under supervision.
That creates an awkward loop. Senior people are needed to verify consequential output, yet companies may weaken the path that produces senior judgment. Keeping every old junior task would be wasteful. Assuming expertise will continue to appear without practice would be equally careless.
The authors call this the Missing Junior Loop. They pair it with the Codifier’s Curse: experienced practitioners help encode their tacit knowledge into systems that reduce demand for the very work through which that experience was acquired. The names are dramatic, but the underlying problem is concrete. An organization can improve short term output while consuming its future supply of judgment.
A better response is to preserve the learning while changing the work. Juniors can investigate failures, construct evaluation sets, compare agent output with real outcomes, and take responsibility for increasingly consequential decisions. The apprenticeship has to become explicit because cheap generation can no longer be relied upon to create it incidentally.
What becomes valuable
If the paper is right, several kinds of work become more important:
- turning an ambiguous goal into a result that can be checked;
- building tests, measurements, provenance, and audit trails;
- deciding which failures are tolerable and which require a person;
- maintaining genuinely independent sources of evidence;
- accepting responsibility when no cheap check can settle the question.
This also changes what a useful AI product should sell. Producing more words, images, or code is easy to demonstrate. A product becomes durable when it can deliver a result with evidence that the customer is willing to trust.
The paper describes the organizational pattern as an AI sandwich: people establish intent, agents perform the scalable execution, and people verify the result. I would treat that as a starting shape rather than a permanent org chart. The important point is that intent and accountability do not disappear merely because the middle becomes cheaper.
The hollow economy and the augmented one
The paper’s negative scenario is a hollow economy. Measured output rises while actual quality becomes harder to see. Organizations optimize the visible proxy, accept more automatically generated work, and discover the missing judgment only after failures have accumulated.
The better scenario is an augmented economy in which verification capacity improves alongside generation. Tests become richer. Provenance follows the work. Independent checks are built into workflows. People spend less time producing routine output and more time deciding what should be produced, investigating exceptions, and accepting responsibility for the result.
A framework, not a forecast
The paper offers a way to reason about automation, not a timetable for which professions disappear. Verification costs can fall too. Better instruments, clearer standards, and new forms of review can turn a judgment problem into a measurable one. Social and legal acceptance may move more slowly than the technology.
Still, the framework catches something that capability benchmarks miss. Producing an answer and knowing that the answer is good are separate economic activities. As producing an answer gets cheaper, verifying it becomes the part worth paying for.