IAB Tech Lab proposes rules around AI content scraping


Publishers have spent the past few years watching their traffic slide — in some cases by more than 50% from search alone — while AI systems ingest and summarize their content at scale. The imbalance has been obvious. The infrastructure to power AI is highly commercialized. The information feeding it is not.

The IAB Tech Lab thinks it’s time to fix that.

Today, the standards body released version 1.0 of its Content Monetization Protocols (COMP) specification for public comment. The idea is straightforward: create a standardized way for publishers and AI systems to establish commercial terms before any crawling or content use happens.

Building payment rails for information

“AI systems require chips, power, and information. Information is the only input in that equation that does not yet have a consistent commercial infrastructure around it,” Anthony Katsur, CEO of IAB Tech Lab, said in a statement. “If we expect high-quality content to continue fueling AI-driven products, we need clear terms of engagement and a mechanism that supports compensation, accountability, and long-term sustainability. CoMP is designed to help the industry move in that direction.”

If AI products are going to keep relying on high-quality journalism and premium content, there needs to be a consistent way to support compensation and accountability. CoMP is designed to create that commercial layer.

Instead of every publisher cutting one-off licensing deals or building custom integrations with each AI platform, CoMP proposes a shared protocol. Content owners could signal permissions and commercial terms in a standardized way. AI systems could check those signals before accessing or using content.

In theory, that reduces friction on both sides.

Not a blockade, but a bridge

CoMP is not meant to replace paywalls, robots.txt files or other access controls. The framework assumes publishers already have strong blocking mechanisms in place at the edge or CDN level.

Your customers search everywhere. Make sure your brand shows up.

The SEO toolkit you know, plus the AI visibility data you need.

Start Free Trial

Get started with

Semrush One Logo

What CoMP does is create a path from “no access” to “access under agreed terms.” It is less about locking doors and more about installing a payment counter next to them.

For AI companies, that could reduce legal and operational risk. For publishers, it opens the door to monetizing AI use more systematically.

This is “an important step toward establishing interoperable, transparent standards for fair value exchange in the AI ecosystem,” Achim Schlosser, VP Global Data Standards at Bertelsmann, said in a statement. “We believe scalable, robust compensation frameworks — alongside visibility and attribution for content usage — are essential to sustaining high-quality journalism and premium content in the AI era.”

Why marketers should care

At first glance, this might sound like a publisher-only story. It is not.

AI answers and agents are increasingly shaping how people discover information and brands. If the economics of content access change, that can ripple through the entire media ecosystem.

If premium publishers find sustainable AI monetization models, the quality and availability of trusted content could stabilize. If they do not, the supply of authoritative content that fuels AI responses could shrink or shift. That matters a lot for brands that rely on earned media, partnerships or high-quality publisher environments.

There is also a bigger structural shift here. We are watching the industry try to formalize an information marketplace for AI. Instead of informal scraping and patchwork licensing, CoMP aims to create common rails.

As AI becomes the intermediary between consumers and content, the business rules around that content are moving front and center. For marketers operating in an AI-shaped discovery landscape, that is not a side story. It is part of the new plumbing of digital media.

The public comment period runs through April 2, 2026. Click here to go to the site.



Source link

Leave a Comment

Your email address will not be published. Required fields are marked *