Simon Willison’s Weblog

Subscribe

Items tagged github, gitscraping in 2020

Filters: Year: 2020 × github × gitscraping × Sorted by date


Git scraping: track changes over time by scraping to a Git repository

Git scraping is the name I’ve given a scraping technique that I’ve been experimenting with for a few years now. It’s really effective, and more people should use it.

[... 963 words]

6:27 pm / 9th October 2020 / git, github, projects, scraping, githubactions, gitscraping

Tracking FARA by deploying a data API using GitHub Actions and Cloud Run

I’m using the combination of GitHub Actions and Google Cloud Run to retrieve data from the U.S. Department of Justice FARA website and deploy it as a queryable API using Datasette.

[... 1599 words]

7:51 am / 21st January 2020 / continuousdeployment, continuousintegration, datajournalism, github, projects, datasette, cloudrun, githubactions, gitscraping

Types

Years

Months

Tags