Software in the Era of AI

Andrej Karpathy’s talk is most useful as a guide to product design: prompts become programs, models become a new computing layer, and people need control over how much work they delegate.

September 2025

Andrej Karpathy’s Software Is Changing (Again) talk gives a name to a transition that is easy to feel and harder to describe. We still write ordinary code. We still train neural networks. We can now also give a general model instructions and examples at runtime, then use its response as part of the program.

Karpathy calls this third mode Software 3.0. The label is memorable, but the product lessons are more important than the taxonomy. A model is capable and inconsistent, so useful software has to assemble its context, constrain its actions, expose its work, and let the user choose how much autonomy is appropriate.

Three ways to create behavior

Karpathy distinguishes three kinds of software by where the program comes from:

A sentiment classifier makes the difference concrete. In ordinary code, a programmer writes rules. In machine learning, a training process fits a model to labeled examples. With a large language model (LLM), the application can describe the task and provide a few examples inside the request.

You are a sentiment classifier.
For each review, answer POSITIVE or NEGATIVE.

Review: The film was beautifully made.
Answer: POSITIVE

Review: The plot never came together.
Answer: NEGATIVE

The prompt is recognizably a program. It defines the role, constrains the output, and supplies examples of the mapping. Unlike the weights of a trained classifier, a person can read and revise it directly. Unlike ordinary source code, its boundaries remain probabilistic.

The third approach is unusually accessible because its programming language is ordinary language. It is also less exact. The prompt does not define behavior with the determinism of source code, and the model may respond differently to inputs that look similar. Software 3.0 therefore joins the other two modes rather than replacing them. Ordinary code remains the right place for strict control. Learned models handle perception and approximation. Prompts provide flexible instructions over the top.

Utility, fab, and operating system

Karpathy tests three analogies for the model business. Models resemble a utility because they require large capital investments, are served centrally, and are metered by usage. They resemble semiconductor fabrication because only a few organizations can afford the training infrastructure and specialized research. Both comparisons explain the economics, but neither describes what developers are building on top.

Karpathy compares frontier models to the operating systems of an early time sharing computer. The expensive model runs centrally. Many users share it over a network. Its context window behaves like working memory, and tools give it access to files, calculators, browsers, and other software.

The analogy is imperfect, as Karpathy acknowledges, but it points toward a useful architecture. The model is one computing layer inside a larger application, surrounded by memory, tools, permissions, and an interface. The application decides which information enters the context, which actions are available, and how a result becomes visible to the user.

This explains why a raw chat window is rarely the final product. A coding tool knows which files are open, which tests failed, and which lines changed. A research tool gathers sources and keeps citations beside its answer. Much of the product value lies in constructing the model’s input and making its output easy to inspect.

The operating system comparison also clarifies the role of tools. The context window is working memory, not permanent storage. A file system provides durable state. A calculator or Python interpreter provides exact computation. A browser provides network access. The model decides when to use those capabilities, but ordinary software still performs the actual reads, calculations, and requests.

This leaves room for very different ecosystems. Some models are available only through the company that trained them. Others can be downloaded, adapted, and run by third parties. That resembles the split between closed operating systems and open ones, although the comparison should not be pushed too far. Model weights are not an operating system kernel, and an LLM provider does not control every application built around its model.

Useful products live between autocomplete and autonomy

Karpathy argues for partial autonomy rather than an immediate jump to unsupervised agents. His phrase is an autonomy slider. At one end, the model suggests a small completion. Farther along, it edits a selected block, changes a file, or works across a repository. At the other end, it pursues a broad goal with little intervention.

Cursor provides his clearest example. A tab completion proposes a few tokens. A selected edit delegates a bounded region. Agent mode can search and modify the repository. Perplexity exposes a similar choice between a quick search and a longer research process. The product lets the user choose a different level of autonomy for each task.

Karpathy describes the interaction as a generation and verification loop. The model produces a draft, plan, or code change. The person checks it and either accepts it, edits it, or asks for another attempt. A useful product improves both halves of that loop. Better context makes the first result more relevant. A purpose built interface makes the result faster to judge.

That interface matters more than a generic chat box suggests. A code diff lets a developer scan additions and deletions directly. A research answer can place its sources beside the claims they support. The product may also use several models behind the scenes, perhaps one to find relevant material, another to reason about it, and another to apply a proposed edit. The user sees one coherent action rather than the plumbing needed to produce it.

The appropriate position depends on the task. A reversible change with a strong test suite can tolerate more autonomy. An ambiguous or consequential decision should remain closer to the person. Trust also changes over time as the user sees how the system behaves in a particular environment.

A good interface makes this delegation legible. It shows what the model intends to do, what it changed, which tools it used, and where it is uncertain. It lets the user interrupt, narrow the scope, or take over. Human judgment belongs where the available checks are weak. Ceremonial clicking adds nothing.

Models have jagged capabilities

One reason the slider matters is that LLM competence is uneven. A model can solve a difficult programming problem and then mishandle a simple comparison. It can recall an obscure concept while accepting a malicious instruction embedded in a document. Its behavior does not fit a single level such as junior, senior, or expert.

The examples in the talk are deliberately mundane. A model with broad technical knowledge may compare decimal numbers incorrectly or count letters in a word badly. The decimal and letter-counting tasks are simple. Their failures matter because they make competence difficult to infer from one impressive result. Success on a demanding task does not establish reliability on the next one.

Karpathy jokingly describes models as “people spirits,” simulations assembled from human text with impressive knowledge and peculiar limitations. The phrase is not a scientific model, but it discourages two bad intuitions. An LLM is not a database that returns a stored fact, and it is not an ordinary program that follows instructions exactly.

Products have to assume that capability varies by task and context. They need external memory because the base model does not learn permanently from one user’s conversation. They need permission boundaries because text can influence behavior. They need verification because fluent output is not evidence of correctness.

Agents become users of software

Most software exposes one interface for people and another for conventional programs. People receive pages and buttons. Programs receive APIs with rigid schemas. Agents sit awkwardly between them. They understand prose and screenshots, but they work more reliably when documentation and actions are explicit.

Karpathy points to plain text documentation, command examples, and machine callable tools as ways to make software easier for agents to use. The important idea is broader than any one file format or protocol. A product now has another kind of user. That user needs concise context, discoverable capabilities, unambiguous actions, and errors that explain how to recover.

He uses llms.txt as one example of agent readable documentation and points to documentation that pairs visual instructions with command line equivalents. “Click the Create Project button” is useful to a person. A concrete command or tool call gives an agent an action it can execute and a result it can inspect.

Karpathy points to Vercel and Stripe as early examples. A plain Markdown version of documentation is easier for a model to read than a page whose structure is mostly visual. A command line example is more reliable than asking an agent to infer which button to press. A tool protocol can go farther by exposing a bounded set of actions with explicit inputs and outputs.

He also mentions tools such as GitIngest and DeepWiki. One turns a repository into a compact body of text that can be placed in a model’s context. The other adds summaries and architectural explanations. Both are context builders. They take information arranged for people or conventional software and reshape it for a model that has limited working memory.

This is one reason the ordinary chores around a small application can still consume more time than generating its code. An agent may write the feature, then struggle with a deployment console, an account setup flow, or documentation written entirely around visual clicks. Making those systems agent legible will remove a different class of friction.

Vibe coding reveals the remaining friction

Karpathy also returns to the phrase vibe coding, which he originally used for building software by describing what he wanted, accepting generated changes, and feeding errors back to the model. In the talk he shows an iOS calorie tracker built without prior Swift experience and MenuGen, a small application that turns a photograph of a menu into pictures of the dishes.

His experience with MenuGen is more informative than the novelty of generating the code. The code was often the easy part. Deployment, API keys, authentication, payments, and configuration still required navigating systems designed around a person clicking through a browser. Software 3.0 made implementation accessible while exposing the amount of Software 1.0 work surrounding it.

That is also why vibe coding has two very different meanings. It can let a newcomer turn an idea into working software. It can also produce a codebase the creator cannot evaluate or maintain. The builder’s ability to verify the result and recover from failure separates the two meanings of vibe coding, regardless of how the code was produced.

The accessibility is real. Karpathy shows children building small applications at an event without first spending years learning a programming language. Software engineering remains necessary as the entry point widens, ideas can be tested sooner, and difficult work moves to different parts of the process. For a disposable experiment, the result may be enough. For software that handles money, private data, or long term maintenance, understanding and verification still matter.

What I kept from the talk

Software 3.0 is a helpful name for code whose behavior is shaped through prompts and context. The deeper point is that nondeterministic capability needs a carefully designed shell.

The best products do four things well. They gather the right context without making the user assemble it by hand. They give the model tools with narrow permissions. They make proposed work easy to verify. They let the user adjust autonomy to match the risk and clarity of the task.

That is a less dramatic future than a fully autonomous agent replacing an application. It is also much closer to the software that is proving useful now.