Simon Willison’s Weblog

Subscribe

September 2020

87 posts: 8 entries, 10 links, 5 quotes, 64 beats

Sept. 8, 2020

Release sqlite-utils 2.18 — Python CLI utility and library for manipulating SQLite databases

Sept. 9, 2020

Release dogsheep-beta 0.7a0 — Build a search index across content from multiple SQLite database tables and run faceted searches against it using Datasette

AVIF has landed. AVIF support landed in Chrome 85 a few weeks ago. It’s a new lossy royalty-free image format derived from AV1 video and it’s really impressive—it can achieve similar results to JPEG using a quarter of the file size! Jake digs into AVIF in detail, providing lots of illustrative examples created using the Squoosh online compressor, which now supports AVIF encoding. Jake used the same WebAssembly encoder from Squoosh to decode AVIF images in a web worker so that the demos in his article would work even for browsers that don’t yet support AVIF natively.

# 4:49 pm / chrome, images, webworkers, webassembly, jake-archibald

Sept. 10, 2020

Release datasette-dns 0.1a0 — Custom SQL function for making DNS lookups
Release datasette-dns 0.1a1 — Custom SQL function for making DNS lookups

15 rules for blogging, and my current streak (via) Matt Webb is on a 24 week streak of blogging multiple posts a week and shares his rules on how he’s doing this. These are really good rules. A rule of thumb that has helped me a lot is to fight back against the temptation to make a post as good as I can before I publish it— because that way lies a giant drafts folder and no actual published content. “Perfect is the enemy of shipped”.

# 6:09 pm / blogging, matt-webb, streaks

Sept. 11, 2020

Stories of reaching Staff-plus engineering roles (via) Extremely useful collection of career stories from staff-level engineers at a variety of different companies, collected by Will Larson.

# 3:30 am / will-larson, careers, management

Weeknotes: datasette-dump, sqlite-backup, talks

I spent some time this week digging into Python’s sqlite3 internals. I also gave two talks and recorded a third, due to air at PyGotham in October.

[... 928 words]

Release datasette-seaborn 0.1a0 — Statistical visualizations for Datasette using Seaborn
Release datasette-seaborn 0.1a1 — Statistical visualizations for Datasette using Seaborn

Sept. 14, 2020

Release datasette 0.49a1 — An open source multi-tool for exploring and publishing data
Release datasette 0.49 — An open source multi-tool for exploring and publishing data
Release datasette-yaml 0.1 — Export Datasette records as YAML
Release datasette-atom 0.8 — Datasette plugin that adds a .atom output format
Release datasette-copyable 0.3 — Datasette plugin for outputting tables in formats suitable for copy and paste
Release datasette-ics 0.5 — Datasette plugin for outputting iCalendar files
TIL Open a debugging shell in GitHub Actions with tmate — > :warning: **17 Feb 2022: There have been reports of running tmate causing account suspensions**. See [this issue](https://github.com/mxschmitt/action-tmate/issues/104) for details. Continue with caution.

Sept. 15, 2020

Release datasette 0.49.1 — An open source multi-tool for exploring and publishing data

“I Have Blood on My Hands”: A Whistleblower Says Facebook Ignored Global Political Manipulation (via) Sophie Zhang worked as the data scientist for the Facebook Site Integrity fake engagement team. She gave up her severance package in order to speak out internally about what she saw there, and someone leaked her memo to BuzzFeed News. It’s a hell of a story: she saw bots and coordinated manual accounts used to influence politics in countries all around the world, and found herself constantly making moderation decisions that had lasting political impact. “With no oversight whatsoever, I was left in a situation where I was trusted with immense influence in my spare time". This sounds like a nightmare—imagine taking on responsibility for protecting democracy in so many different places.

# 9:11 pm / facebook, moderation, social-media

A manager on Strategic Response mused to myself that most of the world outside the West was effectively the Wild West with myself as the part-time dictator – he meant the statement as a compliment, but it illustrated the immense pressures upon me.

Sophie Zhang

# 9:21 pm / facebook

Datasette 0.49: The annotated release notes

Visit Datasette 0.49: The annotated release notes

Datasette 0.49 is out. Some notes on what’s new.

[... 1,234 words]

Sept. 17, 2020

Release github-to-sqlite 2.6 — Save data from GitHub to a SQLite database

Array programming with NumPy—the NumPy paper (via) The NumPy paper is out, published in Nature. I found this enlightening: for an academic paper it’s very understandable, and it filled in quite a few gaps in my mental model of what NumPy is and which problems it addresses, as well as its relationship to the many other tools in the scientific Python stack.

# 4:34 pm / python, scipy, numpy

Release dogsheep-beta 0.7 — Build a search index across content from multiple SQLite database tables and run faceted searches against it using Datasette
Release dogsheep-beta 0.7.1 — Build a search index across content from multiple SQLite database tables and run faceted searches against it using Datasette

Sept. 18, 2020

TIL Talking to a PostgreSQL service container from inside a Docker container — I have a Django application which uses PostgreSQL. I build the Django application into its own Docker container, push that built container to the GitHub package registery and then deploy that container to production.

Weeknotes: datasette-seaborn, fivethirtyeight-polls

Visit Weeknotes: datasette-seaborn, fivethirtyeight-polls

This week I released Datasette 0.49 and tinkered with datasette-seaborn, dogsheep-beta and polling data from FiveThirtyEight.

[... 951 words]

Release dogsheep-beta 0.8 — Build a search index across content from multiple SQLite database tables and run faceted searches against it using Datasette
TIL Turning on Jinja autoescaping when using Template() directly — Jinja autoescaping is turned off by default. Coming from Django this frequently catches me out.

Sept. 19, 2020

TIL Compiling the SQLite spellfix.c module on macOS — I wanted to browse a backup copy of my Plex database, which is a SQLite file. I tried this:

2020 » September

MTWTFSS
 123456
78910111213
14151617181920
21222324252627
282930