From Bots to Brains: The Evolution of Web Scraping

Agentic AI, AI Infrastructure

A fast-paced dive into the history and future of web scraping—from the first chaotic crawlers to modern AI agents.

We’ll explore how scraping evolved, why browser automation is hitting its limits, and how agentic, LLM-driven scrapers are redefining speed and scale for the next generation of data extraction.

Lorenzo Padoan is a Co-founder and CEO of ScrapeGraphAI, an open-source LLM-powered web scraping.

Previously a research engineer at Motion Analytica and research fellow at the University of Padova, he has designed large-scale data systems to analyze human mobility, tourism, and telecom trends across Italy.