Simon Willison on weeknotes

179 items tagged “weeknotes”

2020

Weeknotes: Covid-19, First Python Notebook, more Dogsheep, Tailscale

My covid-19.datasettes.com project publishes information on COVID-19 cases around the world. The project started out using data from Johns Hopkins CSSE, but last week the New York Times started publishing high quality USA county- and state-level daily numbers to their own repository. Here’s the change that added the NY Times data.

[... 993 words]

8:29 pm / 1st April 2020 / teaching, bradfitzpatrick, python, datasette, projects, datajournalism, dogsheep, weeknotes, tailscale, covid19, benwelsh

Weeknotes: Datasette 0.39 and many other projects

This week’s theme: Well, I’m not going anywhere. So a ton of progress to report on various projects.

[... 806 words]

5:33 am / 25th March 2020 / jupyter, sqlite, markdown, datasette, projects, weeknotes, dogsheep, github, fly

Weeknotes: this week was absurd

As of this morning, San Francisco is in a legally mandated shelter-in-place. I can hardly remember what life was like seven days ago. It’s been a very long, very crazy week. This was not a great week for getting stuff done.

[... 246 words]

2:52 am / 18th March 2020 / projects, weeknotes, sqliteutils

Weeknotes: COVID-19 numbers in Datasette

COVID-19, the disease caused by the novel coronavirus, gets more terrifying every day. Johns Hopkins Center for Systems Science and Engineering (CSSE) have been collating data about the spread of the disease and publishing it as CSV files on GitHub.

[... 644 words]

4:49 am / 11th March 2020 / datasette, coronavirus, plugins, projects, weeknotes, covid19

Weeknotes: datasette-ics, datasette-upload-csvs, datasette-configure-fts, asgi-csrf

I’ve been preparing for the NICAR 2020 Data Journalism conference this week which has lead me into a flurry of activity across a plethora of different projects and plugins.

[... 834 words]

2:27 am / 4th March 2020 / csrf, datasettecloud, plugins, search, asgi, ical, security, datasette, projects, datajournalism, weeknotes

Weeknotes: Datasette Writes

As discussed previously, the biggest hole in Datasette’s feature set at the moment involves writing to the database.

[... 604 words]

6:34 am / 26th February 2020 / threads, sqlite, plugins, datasette, csv, weeknotes

Things I learned about shapefiles building shapefile-to-sqlite

The latest in my series of x-to-sqlite tools is shapefile-to-sqlite. I learned a whole bunch of things about the ESRI shapefile format while building it.

[... 1073 words]

5:25 am / 19th February 2020 / shapefiles, sqlite, geojson, projects, spatialite, gis, weeknotes, tommacwright

How to cheat at unit tests with pytest and Black

I’ve been making a lot of progress on Datasette Cloud this week. As an application that provides private hosted Datasette instances (initially targeted at data journalists and newsrooms) the majority of the code I’ve written deals with permissions: allowing people to form teams, invite team members, promote and demote team administrators and suchlike.

[... 933 words]

6:56 am / 11th February 2020 / testing, datasettecloud, python, datasette, unittests, projects, pytest, weeknotes, black

Weeknotes: Shaving yaks for Datasette Cloud

I’ve been shaving a lot of yaks, but I’m finally ready to for other people to start kicking the tires on the MVP of Datasette Cloud.

[... 392 words]

7:34 am / 5th February 2020 / datasettecloud, geojson, datasette, projects, weeknotes

Weeknotes: datasette-auth-existing-cookies and datasette-sentry

Work on Datasette Cloud continues—I’m tantalizingly close to having a MVP I can start to invite people to try out.

[... 701 words]

5:58 am / 29th January 2020 / datasettecloud, cookies, django, datasette, projects, sentry, weeknotes, httpx

Weeknotes: Datasette Cloud and zero downtime deployments

Yesterday’s piece on Tracking FARA by deploying a data API using GitHub Actions and Cloud Run was originally intended to be my weeknotes, but ended up getting a bit too involved.

[... 1431 words]

8:56 pm / 21st January 2020 / docker, traefik, csv, museums, deployment, datasette, projects, weeknotes, datasettecloud, digitalocean, zerodowntime

Weeknotes: Improv at Stanford, planning Datasette Cloud

Last week was the first week of the quarter at Stanford—which is called “shopping week” here because students are expected to try different classes to see which ones they are going to stick with.

[... 806 words]

12:22 am / 14th January 2020 / docker, stanford, jsk, weeknotes, datasettecloud, digitalocean

Building a sitemap.xml with a one-off Datasette plugin

One of the fun things about launching a new website is re-learning what it takes to promote a website from scratch on the modern web. I’ve been thoroughly enjoying using Niche Museums as an excuse to explore 2020-era SEO.

[... 1078 words]

11:02 pm / 6th January 2020 / plugins, seo, museums, datasette, projects, weeknotes

2019

sqlite-utils 2.0: real upserts

I just released version 2.0 of my sqlite-utils library/CLI tool to PyPI.

[... 1140 words]

6:48 am / 30th December 2019 / sql, sqlite, museums, upsert, projects, weeknotes, sqliteutils

Weeknotes: Datasette 0.33

I released Datasette 0.33 yesterday. The release represents an accumulation of small changes and features since Datasette 0.32 back in November. Duplicating the release notes:

[... 678 words]

9:38 pm / 23rd December 2019 / datasette, weeknotes

Logging to SQLite using ASGI middleware

I had some fun playing around with ASGI middleware and logging during our flight back to England for the holidays.

[... 2535 words]

10:30 pm / 16th December 2019 / sqlite, asgi, django, datasette, projects, logging, weeknotes, sqliteutils

Better presentations through storytelling and STAR moments

Last week I completed GSBGEN 315: Strategic Communication at the Stanford Graduate School of Business.

[... 643 words]

12 am / 10th December 2019 / jsk, datasette, speaking, weeknotes

datasette-atom: Define an Atom feed using a custom SQL query

I’ve been having a ton of fun iterating on www.niche-museums.com. I put together some notes on how the site works last week, and I’ve been taking advantage of the Thanksgiving break to continue exploring ways in which Datasette can be used to quickly build database-backed static websites.

[... 1084 words]

12:20 am / 3rd December 2019 / atom, syndication, datasette, projects, weeknotes

niche-museums.com, powered by Datasette

I just released a major upgrade to my www.niche-museums.com website (launched last month).

[... 1154 words]

10:27 pm / 25th November 2019 / projects, datasette, museums, weeknotes, yaml, bakeddata

Weeknotes: datasette-template-sql

Last week I talked about wanting to take ona a larger Datasette project, and listed some candidates. I ended up pushing a big project that I hadn’t listed there: the upgrade of Datasette to Python 3.8, which meant dropping support for Python 3.5 (thanks to incompatible dependencies).

[... 521 words]

10:29 pm / 18th November 2019 / projects, sql, datasette, weeknotes

Weeknotes: Python 3.7 on Glitch, datasette-render-markdown

Streaks is really working well for me. I’m at 12 days of commits to Datasette, 16 posting a daily Niche Museum, 19 of actually reviewing my email inbox and 14 of guitar practice. I rewarded myself for that last one by purchasing an actual classical (as opposed to acoustic) guitar.

[... 1141 words]

11:26 pm / 11th November 2019 / glitch, markdown, museums, datasette, weeknotes, sqliteutils

Weeknotes: More releases, more museums

Lots of small releases this week.

[... 538 words]

8:35 pm / 4th November 2019 / projects, museums, weeknotes

Weeknotes: Niche Museums, Kepler, Trees and Streaks

Every now and then someone will ask “so when are you going to build Museums Near Me then?”, based on my obsession with niche museums and websites like www.owlsnearme.com.

[... 872 words]

10:42 pm / 28th October 2019 / museums, productivity, visualization, projects, weeknotes, bakeddata

Weeknotes: The Squirrel Census, Genome SQL query

This week was mostly about incremental improvements. And squirrels.

[... 911 words]

6:42 pm / 21st October 2019 / squirrels, projects, genetics, weeknotes

Weeknotes: PG&E outages, and Open Source works!

My big focus this week was the PG&E outages project. I’m really pleased with how this turned out: the San Francisco Chronicle used data from it for their excellent PG&E outage interactive (mixing in data on wind conditions) and it earned a bunch of interest on Twitter and some discussion on Hacker News.

[... 452 words]

5 pm / 14th October 2019 / opensource, datasette, projects, dogsheep, weeknotes

Weeknotes: Dogsheep

Having figured out my Stanford schedule, this week I started getting back into the habit of writing some code.

[... 1367 words]

7:33 pm / 7th October 2019 / projects, dogsheep, sqlite, weeknotes

Weeknotes: first week of Stanford classes

One of the benefits of the JSK fellowship is that I can take classes and lectures at Stanford, on a somewhat ad-hoc basis (I don’t take exams or earn credits).

[... 544 words]

4:28 pm / 30th September 2019 / stanford, reading, speaking, music, jsk, weeknotes

Weeknotes: Design thinking for journalists, genome-to-sqlite, datasette-atom

I haven’t had much time for code this week: we’ve had a full five day workshop at JSK with Tran Ha (a JSK alumni) learning how to apply Design Thinking to our fellowship projects and generally to challenges facing journalism.

[... 870 words]

8:13 pm / 20th September 2019 / designthinking, sqlite, reading, genetics, datasette, projects, jsk, weeknotes