Simon Willison’s Weblog

Subscribe

Entries tagged sqlite, scraping

Filters: Type: entry × sqlite × scraping × Sorted by date


git-history: a tool for analyzing scraped data collected using Git and SQLite

I described Git scraping last year: a technique for writing scrapers where you periodically snapshot a source of data to a Git repository in order to record changes to that source over time.

[... 2002 words]

10:32 pm / 7th December 2021 / datajournalism, git, projects, scraping, sqlite, datasette, githistory

Tracking PG&E outages by scraping to a git repo

PG&E have cut off power to several million people in northern California, supposedly as a precaution against wildfires.

[... 833 words]

11:32 pm / 10th October 2019 / datajournalism, projects, scraping, sqlite, datasette, gitscraping, digitalocean, sqliteutils

Types

Years

Tags