Commentary

The Machine Web Needs Its Own Economy

AI is beginning to separate the value of publishing from the act of visiting a publisher.

A piece of journalism can inform an answer in an LLM chat or supply the expertise an agent needs to complete a task. The publisher’s work has created value downstream -- inside an AI product -- without producing a visit to the publisher’s property.

Digital publishing has always depended on some relationship between use and revenue. Traffic made that relationship relatively easy to see. A reader arriving through search could generate advertising revenue or become a subscriber. Publishers built businesses around their ability to turn audience activity into measurable economic value.

AI moves the useful economic event earlier in the chain. When a model retrieves information from a publisher, the reporting may improve the answer before the consumer sees it. As agents take on more research and purchasing decisions, they will consult many sources and present a finished result. Publishers now have another kind of audience: machines consuming information on behalf of people.

advertisement

advertisement

The audience is the market. What's the unit of value?

Cloudflare’s Pay Per Crawl offers the most literal answer. A publisher can set a price for each successful retrieval by an AI crawler, with a minimum of one-tenth of a cent, and vary the price for different content. The transaction occurs at the web’s front door: an unpaid request receives an HTTP 402 “Payment Required” response.

A paid request receives the page. The billable event is access itself.

Access has the virtue of being countable. Its distance from economic value is the problem. A crawler may retrieve an article and discard it. Another article may supply the decisive fact in an answer that guides a purchase or a business decision. Both register as one crawl.

Other experiments are moving the transaction closer to the way the content is used.

Really Simple Licensing -- an open standard supported by publishers including The Associated Press and Vox Media -- lets a site express machine-readable terms for different forms of AI use.

Its specification distinguishes payment for a crawl from payment when content contributes to an AI-generated output. The distinction gives machines a way to discover the commercial terms attached to the information they seek.

The IAB Tech Lab’s Content Monetization Protocols initiative (CoMP) addresses what happens after an agreement exists.

Once an AI system has a license, CoMP provides an API through which it can request content for a stated purpose and receive a machine-readable package. CoMP explicitly leaves the licensing marketplace and economic model to other parties. It can standardize an authorized exchange while leaving the value of that exchange unsettled.

Perplexity has tested a downstream model. Under its Publishers’ Program, a participating publisher shares in advertising revenue when its content is cited in an interaction that generates that revenue. Here, the citation carries economic weight. Payment follows the AI product’s monetization rather than the crawler’s arrival.

Finding a Balance

Taken together, these experiments expose the tradeoffs in choosing a unit of value.

Crawl-based pricing is simple to meter, though it may reward retrieval volume regardless of contribution.

Use-based pricing comes closer to utility and brings a harder attribution question: which source shaped which part of the answer?

Revenue sharing ties publisher compensation to the AI company’s business model, leaving the publisher exposed to the platform’s choices about advertising and attribution.

Bespoke licensing agreements provide another part of the market. They resemble wholesale contracts: large publishers sell access to archives or current reporting under negotiated terms. Those deals establish that machine demand has a price. They serve organizations with enough scale and leverage to negotiate directly, while much of the open web sits outside them.

Machine utility does not follow publisher size. A specialist trade publication may hold the best account of a narrow industry issue, or a local newsroom may own reporting that an agent cannot reliably reconstruct elsewhere. Content with modest human traffic can be unusually valuable when a machine encounters the exact question it answers.

The emerging economy will probably use more than one unit of exchange.

Publishing already supports overlapping markets: an article can help sell a subscription and later be licensed for syndication. Machine use will develop its own versions, shaped by how content contributes to an AI experience.

The hard work is deciding where value becomes visible enough to price. Pricing at retrieval can drift away from utility. Waiting for a citation or commercial outcome requires publishers to trust platforms to measure contribution faithfully. The eventual rules will determine which publishers can participate and what kinds of information remain worth producing.

The web spent decades learning how to price human attention. It now has to learn how to price machine utility. That process has begun. The contest over the billable event will decide how much of the open web has a viable place in the machine economy.

Next story loading loading..