• by Ray Schultz , January 9, 2025

The internet suffered an 86% surge in general invalid traffic (GIVT) in the second half of 2024, driven by AI scrapers and crawlers, according to a study from DoubleVerify’s Fraud Lab. 

GIVT -- defined as self-declared bots used for search engine indexing, stress testing, AI scraping and other such purposes -- is not inherently malicious, the study says.  

But it can distort measurement, inflate impression count and cause discrepancies, it adds. 

AI scrapers were especially active during the election period, causing a GIVT spike.

The top crawlers and scrapings driving known-bot GIVT impressions in December 2024 included Googlebot, MetaAI Crawler, GPT Bot, and AppleBot, Double Verify says.

In addition to the scrapers that were easily identified, there were “more evasive scrapers operated by individuals/organizations who intentionally disguised them as human visitors,” the study continues. “DV continuously identifies non-transparent scrapers and crawlers based on their behavior and digital telemetry.”

The study advises advertisers to optimize their campaign settings, understand platform limitations and address possible GIVT challenges with inclusion lists. 

Why the latter? “Some media-buying partners may advocate for an inclusion-list-only approach or rely primarily on post-bid measurement methods like log file analysis,” the study notes. “These methods fail to proactively address GIVT, especially as new forms emerge in real time.