Simon Willison’s Weblog

Subscribe

TILs

Filters: Sorted by date

TIL Escaping a SQL query to use with curl and Datasette — I used this pattern to pass a SQL query to Datasette's CSV export via curl and output the results, stripping off the first row (the header row) using `tail -n +2`.
8th Dec 2020, 7:05 pm
TIL Controlling the style of dumped YAML using PyYAML — I had a list of Python dictionaries I wanted to output as YAML, but I wanted to control the style of the output.
7th Dec 2020, 8:48 pm
TIL Using custom Sphinx templates on Read the Docs — I wanted to make a small customization to one of my documentation templates on Read the Docs.
7th Dec 2020, 7:07 pm
TIL Running Python code in a subprocess with a time limit — I figured out how to run a subprocess with a time limit for [datasette-ripgrep](https://github.com/simonw/datasette-ripgrep), using the `asyncio.create_subprocess_exec()` method. The pattern looks like this:
7th Dec 2020, 1:03 am
TIL Redirects for Datasette — I made some changes to my https://til.simonwillison.net/ site that resulted in cleaner URL designs, so I needed to setup some redirects. I configured the redirects using a one-off Datasette plugin called `redirects.py` which I dropped into the `plugins/` directory for the Datasette instance:
25th Nov 2020, 7:44 pm
TIL Dropdown menu with details summary — I added dropdown menus to [Datasette 0.51](https://docs.datasette.io/en/stable/changelog.html#v0-51) - see [#1064](https://github.com/simonw/datasette/issues/1064).
1st Nov 2020, 3:12 am
TIL Decorators with optional arguments — [sqlite-utils](https://sqlite-utils.datasette.io/) provides [a decorator](https://sqlite-utils.datasette.io/en/stable/python-api.html#registering-custom-sql-functions) for registering custom Python functions that looks like this:
28th Oct 2020, 9:54 pm
TIL Writing JavaScript that responds to media queries — I wanted to change the layout of [my blog](https://simonwillison.net/) on mobile screens such that the content from the "Elsewhere" right hand column combined with the main column in the correct order (issue [#165](https://github.com/simonw/simonwillisonblog/issues/165)). I couldn't find a way to do this in pure CSS without duplicating a bunch of content, so I decided to do it with JavaScript.
22nd Oct 2020, 3:06 am
TIL Explicit file encodings using click.File — I wanted to add a `--encoding` option to `sqlite-utils insert` which could be used to change the file encoding used to read the incoming CSV or TSV file - see [sqlite-utils #182](https://github.com/simonw/sqlite-utils/issues/182).
16th Oct 2020, 8:39 pm
TIL Upgrading Python Homebrew packages using pip — [VisiData 2.0](https://www.visidata.org/) came out today. I previously installed VisiData using Homebrew, but the VisiData tap has not yet been updated with the latest version.
14th Oct 2020, 4:21 pm
TIL Searching for repositories by topic using the GitHub GraphQL API — I wanted to use the GitHub GraphQL API to return all of the repositories on the https://github.com/topics/git-scraping page.
10th Oct 2020, 2:50 am
TIL Running Datasette on DigitalOcean App Platform — [App Platform](https://www.digitalocean.com/docs/app-platform/) is the new PaaS from DigitalOcean. I figured out how to run Datasette on it.
7th Oct 2020, 2:45 am
TIL Manipulating query strings with URLSearchParams — The `URLSearchParams` class, in [every modern browser](https://caniuse.com/?search=URLSearchParams) since IE 11, provides a sensible API for manipulating query string parameters in JavaScript. I first used it to build Datasette's column action menu, see [table.js](https://github.com/simonw/datasette/blob/0.50a0/datasette/static/table.js) and [issue 981](https://github.com/simonw/datasette/issues/981).
4th Oct 2020, 6:35 pm
TIL Using async/await in JavaScript in Selenium — Thanks [Stuart Langridge](https://twitter.com/sil/status/1312137808111304704) for showing me how to do this:
2nd Oct 2020, 9:19 pm
TIL Installing Selenium for Python on macOS with ChromeDriver — I needed to run Selenium on macOS for the first time today. Here's how I got it working.
2nd Oct 2020, 8:06 pm
TIL Escaping strings in Bash using !:q — TIL this trick, [via Pascal Hirsch](https://twitter.com/phphys/status/1311727268398465029) on Twitter. Enter a line of Bash starting with a `#` comment, then run `!:q` on the next line to see what that would be with proper Bash escaping applied.
1st Oct 2020, 8:32 pm
TIL Figuring out if a text value in SQLite is a valid integer or float — Given a table with a `TEXT` column in SQLite I want to figure out if every value in that table is actually the text representation of an integer or floating point value, so I can decide if it's a good idea to change the type of the column (using [sqlite-utils transform](https://sqlite-utils.datasette.io/en/stable/python-api.html#transforming-a-table)).
27th Sep 2020, 8:40 pm
TIL Understanding option names in Click — I hit [a bug today](https://github.com/simonw/datasette/issues/973) where I had defined a Click option called `open` but in doing so I replaced the Python bulit-in `open()` function:
22nd Sep 2020, 3:49 pm
TIL Compiling the SQLite spellfix.c module on macOS — I wanted to browse a backup copy of my Plex database, which is a SQLite file. I tried this:
19th Sep 2020, 11:11 pm
TIL Turning on Jinja autoescaping when using Template() directly — Jinja autoescaping is turned off by default. Coming from Django this frequently catches me out.
18th Sep 2020, 11:13 pm
TIL Talking to a PostgreSQL service container from inside a Docker container — I have a Django application which uses PostgreSQL. I build the Django application into its own Docker container, push that built container to the GitHub package registery and then deploy that container to production.
18th Sep 2020, 6:43 pm
TIL Open a debugging shell in GitHub Actions with tmate — > :warning: **17 Feb 2022: There have been reports of running tmate causing account suspensions**. See [this issue](https://github.com/mxschmitt/action-tmate/issues/104) for details. Continue with caution.
14th Sep 2020, 10:25 pm
TIL Basic strace to see what a process is doing — I had a long-running process and I wanted to check that it was at least doing _something_.
7th Sep 2020, 11:14 pm
TIL Display EC2 instance costs per month — The [EC2 pricing page](https://aws.amazon.com/ec2/pricing/on-demand/) shows cost per hour, which is pretty much useless. I want cost per month. The following JavaScript, pasted into the browser developer console, modifies the page to show cost-per-month instead.
7th Sep 2020, 2:43 am
TIL Very basic tsc usage — I guess I [have to learn TypeScript](https://twitter.com/simonw/status/1302517496767938561) now.
6th Sep 2020, 4:36 pm
TIL Debugging a Click application using pdb — This tip is for when you are working on a Python command-line application that runs using that program's name, as opposed to typing `python my_script.py`. I usually need this when I'm working on applications built using [Click](https://click.palletsprojects.com/), e.g. projects I start using my [click-app](https://github.com/simonw/click-app) cookiecutter template.
3rd Sep 2020, 5:48 pm
TIL Using the gcloud run services list command — The `gcloud run services list` command lists your services running on Google Cloud Run:
2nd Sep 2020, 2:06 am
TIL Looping over comma-separated values in Bash — Given a file (or a process) that produces comma separated values, here's how to split those into separate variables and use them in a bash script.
2nd Sep 2020, 1:48 am
TIL Start a server in a subprocess during a pytest session — I wanted to start an actual server process, run it for the duration of my pytest session and shut it down at the end.
1st Sep 2020, 2:06 am
TIL Minifying JavaScript with npx uglify-js — While [upgrading CodeMirror](https://github.com/simonw/datasette/issues/948) in Datasette I figured out how to minify JavaScript using `uglify-js` on the command line without first installing any teels, using [npx](https://www.npmjs.com/package/npx) (which downloads and executes a CLI tool while skipping the install step):
30th Aug 2020, 6:20 pm

Years