Fastly Threat Insights Report Reveals Organizations are Fighting Back Against the Rise of the Bots
From AI Crawlers to Headless Bots, Organizations Face New Challenges in Managing Automated Traffic
Fastly, Inc. , a leader in global edge cloud platforms, released its latest Threat Insights Report, offering a detailed analysis of the evolving bot landscape and its implications for businesses. Drawing from trillions of application and API requests across Fastly’s global network, the report highlights how automated traffic is reshaping industries, forcing organizations to adapt their strategies to manage both the risks and opportunities bots present.
The findings reveal that bots now account for nearly one third (29%) of all w**********, with the majority of this bot traffic (about 25% of all w**********) classified as unwanted. Additionally, bot requests increased by 2% compared to the prior quarter. This increase represents billions of requests and is driving organizations to respond, particularly in industries like Media and Entertainment, Commerce, and Financial Services.
Key findings from the report:
- Wanted bots act as a double-edged sword: Organizations are increasingly scrutinizing the value of wanted bots, like AI crawlers and fetchers. While these bots play a critical role in indexing and optimizing content, billions of their requests were blocked (4%). This trend reflects growing concerns about data usage and the impact on revenue, particularly in industries like Media and Entertainment, where AI fetchers are reducing referral traffic to source websites.
- Headless bots become a threat to transaction-heavy industries: Headless bots, which mimic human behavior at machine speed, are targeting transaction-heavy industries like Financial Services and Commerce. 89% of headless bot traffic in Q3 targeted these two industries, reinforcing the need for businesses to implement advanced detection strategies that can distinguish between nefarious headless traffic and intended headless usage so as to avoid disrupting software delivery.
- AI crawlers and fetchers dominate the digital landscape: Meta’s AI crawler and OpenAI’s ChatGPT fetcher emerged as the dominant players, accounting for 60% and 68% of their respective traffic categories. This dominance highlights the growing influence of AI-driven automation on w********** and the critical need for organizations to balance innovation with control over content.
“Fastly’s latest Threat Insights Report highlights the urgency with which organizations should take a proactive approach to bot management as it’s increasingly becoming a critical business issue,” said Marshall Erwin, Chief Information Security Officer at Fastly. “By understanding the nuances of bot traffic, businesses can safeguard their infrastructure, optimize operations, increase brand visibility, and drive innovation.”
The report sheds light on bot industry preferences, and their impact across sectors as organizations reconsider how to remain competitive in the AI era. Findings highlight Commerce (88%) and Public Sector (96%) receiving the highest distribution of crawler traffic, compared to Education (77%) with the highest fetcher volume, suggesting an indication of crawler prioritization or ‘fresh’ yet to be crawled content.
The report also highlights how businesses must adapt with the rise of AI. Media and Entertainment publishers are struggling as AI fetchers scrape their content for generative AI models, leading to diminished referral traffic – only 1% of users click through to source websites that have an AI summary. Commerce saw the highest crawler traffic, driven by demand for real-time inventory and pricing data, underscoring the need to balance automation with competitive protection.
Also Read: The End Of Serendipity: What Happens When AI Predicts Every Choice?
[To share your insights with us, please write to psen@itechseries.com]
Comments are closed.