Simon Willison’s Weblog

Subscribe

TILs

Filters: Sorted by date

TIL Intercepting fetch in a service worker — I'm learning service workers. I wanted to start with one that intercepts calls to a `/path` and returns "Hello World".
None
TIL Atuin for zsh shell history in SQLite — [Atuin](https://github.com/ellie/atuin) (via [Rhet Turnbull](https://twitter.com/RhetTurnbull/status/1518942324004319232)) "replaces your existing shell history with a SQLite database". Obviously I had to try this out!
None
TIL Seeing files opened by a process using opensnoop — I decided to try out [atuin](https://github.com/ellie/atuin?utm_source=tldrnewsletter), a shell extension that writes your history to a SQLite database.
None
TIL Format code examples in documentation with blacken-docs — I decided to enforce that all code examples in the [Datasette documentation](https://docs.datasette.io/) be formatted using [Black](https://github.com/psf/black). Here's [issue 1718](https://github.com/simonw/datasette/issues/1718) where I researched the options for doing this.
None
TIL How to get credentials for a new Twitter bot — I wanted to build a Twitter bot that would tweet Covid sewage graphs from [this page](https://covid19.sccgov.org/dashboard-wastewater) every day.
None
TIL Trick Apple Photos into letting you access your video files — I had an 11GB movie in Apple Photos (sync'd from my iPhone) and I wanted to upload it to YouTube (actually via AirDrop to another laptop first).
None
TIL GUnion to combine geometries in SpatiaLite — I was playing around with [datanews/amtrak-geojson](https://github.com/datanews/amtrak-geojson), which provides GeoJSON for Amtrak stations and track segments.
None
TIL Using awk to add a prefix — I wanted to dynamically run the following command against all files in a directory:
None
TIL Logging users out of Auth0 — If you [implement Auth0](https://til.simonwillison.net/auth0/oauth-with-auth0) for login, you may be tempted to skip implementing logout. I started out just with a `/logout/` page that cleared my own site's cookies, ignoring the Auth0 side of it.
None
TIL Treating warnings as errors in pytest — I was seeing this warning in a Django project when I thought I was correctly using timezone-aware dates everywhere:
None
TIL Allowing a container in Docker Desktop for Mac to talk to a PostgreSQL server on the host machine — I like using [Postgres.app](https://postgresapp.com/) to run PostgreSQL on my macOS laptop. I use it for a bunch of different projects.
None
TIL Generating a calendar week grid with the Python Calendar module — I needed to generate a grid calendar that looks like this (design [by Natalie Downe](https://github.com/natbat/pillarpointstewards/issues/23)):
None
TIL Compiling to WASM with llvm on macOS — [howto-wasm-minimal](https://github.com/ern0/howto-wasm-minimal) by Zalka Ernő ([my fork here](https://github.com/simonw/howto-wasm-minimal)) is a neat demo of a minimal WASM module. It uses C++ to define functions for simple image manipulation including blurring an image, compiles it to WASM using llvm/clang++, then uses JavaScript to run those functions against an image loaded into a `<canvas>` element.
None
TIL Deploying a live Datasette demo when the tests pass — I've implemented this pattern a bunch of times now - here's the version I've settled on for my [datasette-auth0 plugin](https://github.com/simonw/datasette-auth0) repository.
None
TIL Simplest possible OAuth authentication with Auth0 — [Auth0](https://auth0.com/) provides an authentication API which you can use to avoid having to deal with user accounts in your own web application.
None
TIL Extracting web page content using Readability.js and shot-scraper — [Readability.js](https://github.com/mozilla/readability) is "A standalone version of the readability library used for Firefox Reader View".
None
TIL Exporting and editing a Twitter Spaces recording — I hosted [a Twitter Spaces conversation](https://twitter.com/simonw/status/1504604448202518529) the other day. The recording is only available for 30 days afterwards, and I wanted to publish it elsewhere.
None
TIL Rewriting a repo to contain the history of just specific files — I wanted to start [a new git repository](https://github.com/simonw/graphql-scraper/tree/828a1efc4307cca6cd378c394c2d33eac2eceb52) containing just the history of two specific files from my [help-scraper repository](https://github.com/simonw/help-scraper).
None
TIL Using the GitHub Actions cache with npx and no package.json — Some of my repositories have GitHub Actions workflows that execute commands using `npx`, for example my [graphql-scraper](https://github.com/simonw/graphql-scraper) repo using `npx` to [install and run](https://github.com/simonw/help-scraper/blob/54ef9b1fa579dc2d3f68055ffdac9996fd6dae9c/.github/workflows/scrape.yml#L76-L80) the `get-graphql-schema` tool:
None
TIL Counting SQLite virtual machine operations — When SQLite executes a query, it does so by executing a sequence of virtual machine operations.
None
TIL The simplest recursive CTE — I found this really simple recursive CTE useful for ensuring I understood how to write recursive CTEs.
None
TIL Async fixtures with pytest-asyncio — I wanted to use a fixture with `pytest-asyncio` that was itsef as `async def` function, so that it could execute `await` statements.
None
TIL Concatenating strings and newlines in Google Sheets — I was asked if there was a way to run [shot-scraper](https://github.com/simonw/shot-scraper) against a list of URLs in a Google Sheet.
None
TIL pytest coverage with context — [This tweet](https://twitter.com/mariatta/status/1499863816489734146) from \@Mariatta tipped me off to the ability to measure "contexts" when [running coverage](https://coverage.readthedocs.io/en/6.3.2/contexts.html#context-reporting) - as a way to tell which tests exercise which specific lines of code.
None
TIL Natural Earth in SpatiaLite and Datasette — Natural Earth ([website](https://www.naturalearthdata.com/), [Wikipedia](https://en.wikipedia.org/wiki/Natural_Earth)) is a a public domain map dataset.
None
TIL Installing Python on macOS with the official Python installer — I usually use Homebrew on macOS, but I decided to try using the official Python installer based on [this Twitter conversation](https://twitter.com/rtpg_/status/1498115527465914371).
None
TIL Running the latest SQLite in Datasette using Homebrew — I made a pleasant discovery today: Homebrew are very quick to update to the latest SQLite release (here's [their formula](https://github.com/Homebrew/homebrew-core/blob/master/Formula/sqlite.rb)), and since [Datasette](https://datasette.io/) when installed via Homebrew uses that version, this means you can use `brew update sqlite` to ensure you are running the most recent SQLite version within Datasette.
None
TIL Crawling Datasette with Datasette — I wanted to add the new tutorials on https://datasette.io/tutorials to the search index that is used by the https://datasette.io/-/beta search engine.
None
TIL Using curl to run GraphQL queries from the command line — I wanted to run a query against the GitHub GraphQL API using `curl` on the command line, while keeping the query itself as readable as possible. Here's the recipe I came up with (tested in both `bash` and `zsh`), with TOKEN replaced by my GitHub API personal access token:
None
TIL Recursively fetching metadata for all files in a Google Drive folder — For [google-drive-to-sqlite](https://github.com/simonw/google-drive-to-sqlite) I wanted a mechanism to recursively return metadata on every file in a specified Goole Drive folder.
None