What is currently the best technology stack for web scraping?
10th December 2013
My answer to What is currently the best technology stack for web scraping? on Quora
PhantomJS combined with CasperJS is pretty fantastic—it runs a full, headless copy of a Webkit browser so it can operate against a real DOM, execute JavaScript properly, even grab full rendered screenshots of areas of the page but is still easy to automate.More recent articles
- Highlights from my appearance on the Data Renegades podcast with CL Kao and Dori Wilson - 26th November 2025
- Claude Opus 4.5, and why evaluating new LLMs is increasingly difficult - 24th November 2025
- sqlite-utils 4.0a1 has several (minor) backwards incompatible changes - 24th November 2025