Simon Willison’s Weblog

Subscribe

March 2022

74 posts: 9 entries, 10 links, 2 quotes, 53 beats

March 1, 2022

Release datasette-dateutil 0.3 — dateutil functions for Datasette
Release xml-analyser 1.3 — Simple command line tool for quickly analysing the structure of an arbitrary XML file

March 2, 2022

Release sqlite-utils 3.25 — Python CLI utility and library for manipulating SQLite databases

March 4, 2022

Release geojson-to-sqlite 1.0 — CLI tool for converting GeoJSON files to SQLite (with SpatiaLite)

Working with the web platform is dealing with history, with the accumulated matter of quirksmode and good-enough standards. In exchange for the ability to deliver instantly-updating software directly to customers with no middlemen and no installation, you have to absorb a great deal of nearly-useless information that’s entirely about dodging meaningless traps.

Tom MacWright

# 4:11 pm / web, tom-macwright

None
TIL Natural Earth in SpatiaLite and Datasette — Natural Earth ([website](https://www.naturalearthdata.com/), [Wikipedia](https://en.wikipedia.org/wiki/Natural_Earth)) is a a public domain map dataset.
Sighting 12:31 PM – 12:37 PM — Brewer's Blackbird, California Gull
Brewer's Blackbird
Brewer's Blackbird
California Gull
California Gull

March 5, 2022

None
TIL pytest coverage with context — [This tweet](https://twitter.com/mariatta/status/1499863816489734146) from \@Mariatta tipped me off to the ability to measure "contexts" when [running coverage](https://coverage.readthedocs.io/en/6.3.2/contexts.html#context-reporting) - as a way to tell which tests exercise which specific lines of code.

Why I invented “dash encoding”, a new encoding scheme for URL paths

Visit Why I invented "dash encoding", a new encoding scheme for URL paths

Datasette now includes its own custom string encoding scheme, which I’ve called dash encoding. I really didn’t want to have to invent something new here, but unfortunately I think this is the best solution to my very particular problem. Some notes on how dash encoding works and why I created it.

[... 1,392 words]

March 8, 2022

Release django-sql-dashboard 1.0.2 — Django app for building dashboards using raw SQL queries

lite-youtube-embed (via) Handy Web Component wrapper around the standard YouTube iframe embed which knocks over 500KB of JavaScript off the initial page load—I just added this to the datasette.io homepage and increased the Lighthouse performance score from 51 to 93!

# 9:13 pm / iframes, paul-irish, youtube, web-performance, web-components

Release shot-scraper 0.1a0 — A command-line utility for taking automated screenshots of websites

March 9, 2022

Release shot-scraper 0.1 — A command-line utility for taking automated screenshots of websites

Postgres Auditing in 150 lines of SQL (via) I’ve run up against the problem of tracking changes made to rows within a database table so many times, and I still don’t have a preferred solution. This approach to it looks very neat: it uses PostgreSQL triggers to populate a single audit table (as opposed to one audit table per tracked table) and records the previous and current column values for the row using jsonb.

# 7:19 pm / databases, postgresql

Release shot-scraper 0.2 — A command-line utility for taking automated screenshots of websites
Release shot-scraper 0.3 — A command-line utility for taking automated screenshots of websites

March 10, 2022

shot-scraper: automated screenshots for documentation, built on Playwright

Visit shot-scraper: automated screenshots for documentation, built on Playwright

shot-scraper is a new tool that I’ve built to help automate the process of keeping screenshots up-to-date in my documentation. It also doubles as a scraping tool—hence the name—which I picked as a complement to my git scraping and help scraping techniques.

[... 1,802 words]

Sighting 11:06 AM – 11:34 AM — California Sea Lion, Willet
California Sea Lion
California Sea Lion
Willet
Willet

curlconverter.com (via) This is pretty magic: paste in a “curl” command (including the ones you get from browser devtools using copy-as-curl) and this will convert that into code for making the same HTTP request... using Python, JavaScript, PHP, R, Go, Rust, Elixir, Java, MATLAB, Ansible URI, Strest, Dart or JSON.

# 8:12 pm / curl, http

Release shot-scraper 0.4 — A command-line utility for taking automated screenshots of websites

March 11, 2022

Release sqlite-utils 3.25.1 — Python CLI utility and library for manipulating SQLite databases
Release image-diff 0.1 — CLI tool for comparing images
Release image-diff 0.2 — CLI tool for comparing images
Release image-diff 0.2.1 — CLI tool for comparing images

March 12, 2022

Weeknotes: Distracted by Playwright

Visit Weeknotes: Distracted by Playwright

My goal for this week was to unblock progress on Datasette by finally finishing the dash encoding implementation I described last week. I was getting close, and then I got very distracted by Playwright.

[... 892 words]

Release shot-scraper 0.5 — A command-line utility for taking automated screenshots of websites

@newshomepages (via) Ben Welsh used my shot-scraper tool and GitHub Actions to launch a Twitter bot which tweets screenshots of newspaper homepages on a scheduled basis. Ben says: “The tech is so easy, I was able to pull it off in a couple hours at zero cost. A decade ago I ran a similar project using the cloud resources of the day. [...] It costs thousands of dollars and the screenshots were of much lower quality. Incredible progress!”

# 7:21 pm / twitter, github-actions, playwright, shot-scraper, ben-welsh

Release shot-scraper 0.6 — A command-line utility for taking automated screenshots of websites

March 13, 2022

Release shot-scraper 0.7 — A command-line utility for taking automated screenshots of websites
Release shot-scraper 0.8 — A command-line utility for taking automated screenshots of websites

2022 » March

MTWTFSS
 123456
78910111213
14151617181920
21222324252627
28293031