Weeknotes: PG&E outages, and Open Source works!
14th October 2019
My big focus this week was the PG&E outages project. I’m really pleased with how this turned out: the San Francisco Chronicle used data from it for their excellent PG&E outage interactive (mixing in data on wind conditions) and it earned a bunch of interest on Twitter and some discussion on Hacker News.
I gave a talk about the project on Thursday for the Online News Association Silicon Valley meetup and made connections with a bunch of interesting journalists from around the Bay Area.
Open Source works!
A highlight of last week was that a bunch of my projects gained new functionality through open source contributions!
- I landed three pull requests to Datasette from Tobias Kunze—a metadata fix, smarter database ordering and a long-wanted button to auto-indent SQL queries.
- datasette-leaflet-json is a plugin that I built last year but never promoted because it had a frustrating bug in it. Chris Shaw spotted and fixed the bug, and bumped the Leaflet version dependency too. These changes are now shipped as version 0.3.
- I quietly shipped a new plugin, datasette-render-timestamps, last week which identifies columns containing Unix-style timestamp integers and renders them as a readable string. Chris spotted that too, and added a feature to let you configure the formatting in a
metadata.jsonplugin setting. It was the best kind of pull request, incorporating the fix, the tests and the documentation update all in a single commit.
- Ishan Anand plans to use datasette-auth-github to hook into nginx authentication. He submitted a PR to refactor the
GithubAuthclass to support building that as a separate project.
I started a very basic website for my Dogsheep personal analytics project.
I also started running various Dogsheep tools via cron on my personal Dogsheep server, to keep that set of databases automatically updated with my latest activity on various services.
Most excitingly, Tobias Kunze built the first independent Dogsheep-style tool: goodreads-to-sqlite!
As the name suggests, it imports your data from Goodreads into a SQLite database. It inspired me to create a Goodreads account which I will be using to track my book reading activity from now on.
Tobias wrote a fantastic blog post introducing the tool which includes some neat example queries and graphs.
In other Dogsheep news, I added an issue-comments command to github-to-sqlite for fetching all issue comments in a repo. My goal is to evolve that tool to the point where it can import all relevant data from all of my repositories and give me a single Datasette-powered dashboard for keeping track of everything in one place.
More recent articles
- Datasette Enrichments: a new plugin framework for augmenting your data - 1st December 2023
- llamafile is the new best way to run a LLM on your own computer - 29th November 2023
- Prompt injection explained, November 2023 edition - 27th November 2023
- I'm on the Newsroom Robots podcast, with thoughts on the OpenAI board - 25th November 2023
- Weeknotes: DevDay, GitHub Universe, OpenAI chaos - 22nd November 2023
- Deciphering clues in a news article to understand how it was reported - 22nd November 2023
- Exploring GPTs: ChatGPT in a trench coat? - 15th November 2023
- Financial sustainability for open source projects at GitHub Universe - 10th November 2023
- ospeak: a CLI tool for speaking text in the terminal via OpenAI - 7th November 2023
- DALL-E 3, GPT4All, PMTiles, sqlite-migrate, datasette-edit-schema - 30th October 2023