Simon Willison’s Weblog

Subscribe

TILs

Filters: Sorted by date

TIL Using boto3 from the command line — I found a useful pattern today for automating more complex AWS processes as pastable command line snippets, using [Boto3](https://aws.amazon.com/sdk-for-python/).
3rd Aug 2022, 3:34 am
TIL Related content with SQLite FTS and a Datasette template function — Today I added "related TILs" to this TIL website - so each TIL now shows five related TILs at the bottom of the page.
1st Aug 2022, 12:53 am
TIL Emulating a big-endian s390x with QEMU — I got [a bug report](https://github.com/simonw/sqlite-fts4/issues/6) concerning my [sqlite-fts4](https://github.com/simonw/sqlite-fts4) project running on PPC64 and s390x architectures.
30th Jul 2022, 2:44 am
TIL Migrating a GitHub wiki from one repository to another — I figured out how to migrate a [GitHub wiki](https://docs.github.com/en/communities/documenting-your-project-with-wikis/about-wikis) (public or private) from one repository to another while preserving all history.
28th Jul 2022, 8:08 pm
TIL struct endianness in Python — TIL the Python standard library [struct](https://docs.python.org/3/library/struct.html) module defaults to interpreting binary strings using the endianness of your machine.
28th Jul 2022, 3:48 pm
TIL Testing things in Fedora using Docker — I got [a report](https://twitter.com/peterjanes/status/1552407491819884544) of a bug with my [s3-ocr tool](https://simonwillison.net/2022/Jun/30/s3-ocr/) running on Fedora.
27th Jul 2022, 10:41 pm
TIL Deploying a redbean app to Fly — [redbean](https://redbean.dev/) is a fascinating project - it provides a web server in a self-contained executable which you can add assets (or dynamic Lua code) to just by zipping them into the same binary package.
25th Jul 2022, 12:46 am
TIL Using pytest and Playwright to test a JavaScript web application — I [decided to add](https://github.com/simonw/datasette-lite/issues/35) automated tests to my [Datasette Lite](https://simonwillison.net/2022/May/4/datasette-lite/) project. Datasette Lite bundles my Datasette Python web application as a client-side application running inside WebAssembly using Pyodide.
24th Jul 2022, 6:47 pm
TIL Freezing requirements with pip-tools — I tried [pip-tools](https://github.com/jazzband/pip-tools) for the first time today to pin the requirements for the [natbat/pillarpointstewards](https://github.com/natbat/pillarpointstewards) Django app.
14th Jul 2022, 6:22 pm
TIL Testing Electron apps with Playwright and GitHub Actions — Yesterday [I figured out (issue 133)](https://github.com/simonw/datasette-app/issues/133) how to use Playwright to run tests against my Electron app, and then execute those tests in CI using GitHub Actions, for my [datasett-app](https://github.com/simonw/datasette-app) repo for my [Datasette Desktop](https://datasette.io/desktop) macOS application.
13th Jul 2022, 10:29 pm
TIL Upgrading packages with npm — There's a new version of [Vite](https://vitejs.dev/) out (3.0) and I wanted to upgrade my [datasette-table](https://github.com/simonw/datasette-table) package to use it.
13th Jul 2022, 3:04 pm
TIL Conditionally running a second job in a GitHub Actions workflow — My [simonwillisonblog-backup workflow](https://github.com/simonw/simonwillisonblog-backup/blob/main/.github/workflows/backup.yml) periodically creates a JSON backup of my blog's PostgreSQL database, using [db-to-sqlite](https://datasette.io/tools/db-to-sqlite) and [sqlite-diffable](https://datasette.io/tools/sqlite-diffable). It then commits any changes back to the repo using this pattern:
11th Jul 2022, 8:39 pm
TIL Reporting bugs in GitHub to GitHub — I found out today (via [this post](https://github.com/github-community/community/discussions/19988)) about a dedicated interface for reporting bugs in GitHub to GitHub:
8th Jul 2022, 5:14 am
TIL Passing command arguments using heredoc syntax — This trick works for both Bash and zsh.
7th Jul 2022, 10:12 pm
TIL Running OCR against a PDF file with AWS Textract — [Textract](https://aws.amazon.com/textract/) is the AWS OCR API. It's very good - I've fed it hand-written notes from the 1890s and it read them better than I could.
28th Jun 2022, 7:32 pm
TIL Ignoring errors in a section of a Bash script — For [simonw/museums#32](https://github.com/simonw/museums/issues/32) I wanted to have certain lines in my Bash script ignore any errors: lines that used `sqlite-utils` to add columns and configure FTS, but that might fail with an error if the column already existed or FTS had already been configured.
28th Jun 2022, 12:24 am
TIL One-liner for running queries against CSV files with SQLite — I figured out how to run a SQL query directly against a CSV file using the `sqlite3` command-line utility:
20th Jun 2022, 11 pm
TIL Registering new Datasette plugin hooks by defining them in other plugins — I'm experimenting with a Datasette plugin that itself adds new plugin hooks which other plugins can then interact with.
17th Jun 2022, 8:04 pm
TIL Using iconv to convert the text encoding of a file — In [sqlite-utils issue 439](https://github.com/simonw/sqlite-utils/issues/439) I was testing against a CSV file that used UTF16 little endian encoding, also known as `utf-16-le`.
14th Jun 2022, 10:42 pm
TIL Formatting thousands in Jinja — Here's how to format a number in Jinja with commas for thousands, without needing any custom filters or template functions:
8th Jun 2022, 11:34 pm
TIL Making HTTP calls using IPv6 — Tiny TIL today: I learned how to make an HTTP call to an IPv6 address. The trick is to enclose the address in the URL in square braces:
8th Jun 2022, 10:58 pm
TIL Using just with Django — Jeff Triplett [convinced me](https://twitter.com/webology/status/1532860591307726851) to take a look at [just](https://github.com/casey/just) as a command automation tool - sort of an alternative to Make, except with a focus on commands rather than managing build dependencies.
6th Jun 2022, 9:24 pm
TIL Wildcard DNS and SSL on Fly — [Fly](https://fly.io/) makes it surprisingly easy to configure wildcard DNS, such that `anything.your-new-domain.dev` is served by a single Fly application (which can include multiple instances in multiple regions with global load-balancing).
26th May 2022, 2:46 am
TIL Writing Fly logs to S3 — [Fly](https://fly.io/) offers [fly-log-shipper](https://github.com/superfly/fly-log-shipper) as a container you can run in a Fly application to send all of the logs from your other applications to a logging provider.
25th May 2022, 7:47 pm
TIL Using the Fly Docker registry — [Fly.io](https://fly.io/) lets you deploy Docker containers that will be compiled as a Firecracker VM and run in locations around the world.
22nd May 2022, 2:33 am
TIL Optimizing PNGs in GitHub Actions using Oxipng — My [datasette-screenshots](https://github.com/simonw/datasette-screenshots) repository generates screenshots of [Datasette](https://datasette.io/) using my [shot-scraper](https://github.com/simonw/shot-scraper) tool, for people who need them for articles or similar.
19th May 2022, 1:45 am
TIL GitHub Actions job summaries — New feature [announced here](https://github.blog/2022-05-09-supercharging-github-actions-with-job-summaries/). Here's the [full documentation](https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#adding-a-job-summary).
17th May 2022, 5:28 pm
TIL Efficiently copying a file — **TLDR:** Use `shutil.copyfileobj(fsrc, fdst)`
14th May 2022, 12:22 am
TIL Setting up a custom subdomain for a GitHub Pages site — This is so much easier than I was expecting it to be.
4th May 2022, 10:25 pm
TIL Intercepting fetch in a service worker — I'm learning service workers. I wanted to start with one that intercepts calls to a `/path` and returns "Hello World".
30th Apr 2022, 10:56 pm

Years