Grabbing web pages with Perl and PHP
1st September 2002
Web Basics with LWP (via Scott) is an excellent tutorial on Perl’s LWP, a powerful set of modules which make it easy to retrieve content from the web. I’ve been using the excellent Snoopy class for PHP for the same purpose, but I have to admit it isn’t half as comprehensive as LWP. I’ve also written my own simple function safeGet for more light weight tasks—it grabs and returns the contents of a web page but limits both the size of the page and the maximum time it can take to download it.
More recent articles
- Weeknotes: Llama 3, AI for Data Journalism, llm-evals and datasette-secrets - 23rd April 2024
- Options for accessing Llama 3 from the terminal using LLM - 22nd April 2024
- AI for Data Journalism: demonstrating what we can do with this stuff right now - 17th April 2024
- Three major LLM releases in 24 hours (plus weeknotes) - 10th April 2024
- Building files-to-prompt entirely using Claude 3 Opus - 8th April 2024
- Running OCR against PDFs and images directly in your browser - 30th March 2024
- llm cmd undo last git commit - a new plugin for LLM - 26th March 2024
- Building and testing C extensions for SQLite with ChatGPT Code Interpreter - 23rd March 2024
- Claude and ChatGPT for ad-hoc sidequests - 22nd March 2024
- Weeknotes: the aftermath of NICAR - 16th March 2024