Simon Willison’s Weblog

Subscribe

June 2021

54 posts: 5 entries, 17 links, 3 quotes, 29 beats

June 1, 2021

I’m pretty convinced that the biggest single contributor to improved software in my lifetime wasn’t object-orientation or higher-level languages or functional programming or strong typing or MVC or anything else: It was the rise of testing culture.

Tim Bray

# 2:35 pm / testing, tim-bray

June 3, 2021

Weeknotes: Docker architectures, sqlite-utils 3.7, nearly there with Datasette 0.57

This week I learned a whole bunch about using Docker to emulate different architectures, released sqlite-utils 3.7 and made a ton of progress towards the almost-ready-to-ship Datasette 0.57.

[... 1,081 words]

Release sqlite-utils 3.8 — Python CLI utility and library for manipulating SQLite databases
TIL Building a Markdown summary of Django group permissions — [Django SQL Dashboard](https://django-sql-dashboard.datasette.io/) can display rendered markdown generated by a SQL query.
Release django-sql-dashboard 0.15.1 — Django app for building dashboards using raw SQL queries

June 5, 2021

Release datasette 0.56.1 — An open source multi-tool for exploring and publishing data
Release datasette 0.57 — An open source multi-tool for exploring and publishing data

Datasette 0.57. Released today, Datasette 0.57 has new options for controlling which columns are visible on a table page, a way to show more than the default 30 facet results, a whole bunch of smaller improvements and a fix for a severe cross-site scripting security vulnerability.

# 11:12 pm / projects, datasette

Reflected cross-site scripting issue in Datasette (via) Here’s the GitHub security advisory I published for the XSS hole in Datasette. The fix is available in versions 0.57 and 0.56.1, both released today.

# 11:14 pm / security, xss, datasette

Release datasette-auth-passwords 0.4.1 — Datasette plugin for authentication using passwords

June 6, 2021

The humble hash aggregate (via) Today I learned that “hash aggregate” is the name for the algorithm where you split a list of tuples on a common key, run an aggregation against each resulting group and combine the results back together again—I’d previously thought if this in terms of map/reduce but hash aggregate is a much older term used widely by SQL engines—I’ve seen it come up in PostgreSQL explain query output (for GROUP BY) before but didn’t know what it meant.

# 4:03 pm / algorithms, mapreduce, sql

Release django-sql-dashboard 0.16 — Django app for building dashboards using raw SQL queries
TIL Exporting Amplitude events to SQLite — [Amplitude](https://amplitude.com/) offers an "Export Data" button in the project settings page. This can export up to 365 days of events (up to 4GB per export), where the export is a zip file containing `*.json.gz` gzipped newline-delimited JSON.

Apple’s tightly controlled App Store is teeming with scams. I’m quoted in an article in the Washington Post today (linked at the top of the homepage!) explaining how I got scammed on the App Store and spent $19 on a TV remote app with a similar name to the official Samsung app. I mistakenly assumed that the App Store review process wouldn’t allow an app called “Smart Things” to show up in search when I was looking for SmartThings, the official name—and assumed that Samsung were nickel-and-diming their customers rather than expecting the App Store review process to have failed so obviously.

# 10:13 pm / appstore, scams, washington-post, press-quotes

An incomplete list of skills senior engineers need, beyond coding. By Camille Fournier, author of my favourite book on engineering management “The Manager’s Path”. Number one is “How to run a meeting, and no, being the person who talks the most in the meeting is not the same thing as running it”.

# 10:17 pm / careers, management, camillefournier

June 7, 2021

Release xml-analyser 1.1 — Simple command line tool for quickly analysing the structure of an arbitrary XML file
Release datasette-upload-csvs 0.6.1 — Datasette plugin for uploading CSV files and converting them to database tables
Release xml-analyser 1.2 — Simple command line tool for quickly analysing the structure of an arbitrary XML file

June 8, 2021

Release datasette 0.57.1 — An open source multi-tool for exploring and publishing data

When I was a performance consultant I'd show up to random companies who wanted me to fix their computer performance issues. If they trusted me with a login to their production servers, I could help them a lot quicker. To get that trust I knew which tools looked but didn't touch: Which were observability tools and which were experimental tools. "I'll start with observability tools only" is something I'd say at the start of every engagement.

Brendan Gregg

# 7:33 pm / performance, observability, brendan-gregg

June 9, 2021

I saw millions compromise their Facebook accounts to fuel fake engagement. Sophie Zhang, ex-Facebook, describes how millions of Facebook users have signed up for “autolikers”—programs that promise likes and engagement for their posts, in exchange for access to their accounts which are then combined into the larger bot farm and used to provide likes to other posts. “Self-compromise was a widespread problem, and possibly the largest single source of existing inauthentic activity on Facebook during my time there. While actual fake accounts can be banned, Facebook is unwilling to disable the accounts of real users who share their accounts with a bot farm.”

# 3:40 pm / facebook, social-media

June 10, 2021

Release datasette-placekey 0.1 — SQL functions for working with placekeys
Release datasette 0.58a0 — An open source multi-tool for exploring and publishing data
Release datasette-mask-columns 0.2.1 — Datasette plugin that masks specified database columns

June 11, 2021

TIL Running a MySQL server using Homebrew — First, install MySQL like so:
Release db-to-sqlite 1.4 — CLI tool for exporting tables or queries from any SQL database to a SQLite file

June 12, 2021

Release sqlite-utils 3.9 — Python CLI utility and library for manipulating SQLite databases

Weeknotes: New releases across nine different projects

A new release and security patch for Datasette, plus releases of sqlite-utils, datasette-auth-passwords, django-sql-dashboard, datasette-upload-csvs, xml-analyser, datasette-placekey, datasette-mask-columns and db-to-sqlite.

[... 861 words]

Best Practices Around Production Ready Web Apps with Docker Compose (via) I asked on Twitter for some tips on Docker Compose and was pointed to this article by Nick Janetakis, which has a whole host of useful tips and patterns I hadn’t encountered before.

# 2:36 am / docker

June 13, 2021

Release sqlite-utils 3.9.1 — Python CLI utility and library for manipulating SQLite databases

2021 » June

MTWTFSS
 123456
78910111213
14151617181920
21222324252627
282930