What is currently the best technology stack for web scraping?
10th December 2013
My answer to What is currently the best technology stack for web scraping? on Quora
PhantomJS combined with CasperJS is pretty fantastic—it runs a full, headless copy of a Webkit browser so it can operate against a real DOM, execute JavaScript properly, even grab full rendered screenshots of areas of the page but is still easy to automate.More recent articles
- Designing agentic loops - 30th September 2025
- Claude Sonnet 4.5 is probably the "best coding model in the world" (at least for now) - 29th September 2025
- I think "agent" may finally have a widely enough agreed upon definition to be useful jargon now - 18th September 2025