Entries
Filters: Sorted by date
niche-museums.com, powered by Datasette
I just released a major upgrade to my www.niche-museums.com website (launched last month).
[... 1,154 words]Weeknotes: datasette-template-sql
Last week I talked about wanting to take ona a larger Datasette project, and listed some candidates. I ended up pushing a big project that I hadn’t listed there: the upgrade of Datasette to Python 3.8, which meant dropping support for Python 3.5 (thanks to incompatible dependencies).
[... 521 words]Weeknotes: Python 3.7 on Glitch, datasette-render-markdown
Streaks is really working well for me. I’m at 12 days of commits to Datasette, 16 posting a daily Niche Museum, 19 of actually reviewing my email inbox and 14 of guitar practice. I rewarded myself for that last one by purchasing an actual classical (as opposed to acoustic) guitar.
[... 1,141 words]Weeknotes: Niche Museums, Kepler, Trees and Streaks
Every now and then someone will ask “so when are you going to build Museums Near Me then?”, based on my obsession with niche museums and websites like www.owlsnearme.com.
[... 872 words]Weeknotes: The Squirrel Census, Genome SQL query
This week was mostly about incremental improvements. And squirrels.
[... 911 words]Weeknotes: PG&E outages, and Open Source works!
My big focus this week was the PG&E outages project. I’m really pleased with how this turned out: the San Francisco Chronicle used data from it for their excellent PG&E outage interactive (mixing in data on wind conditions) and it earned a bunch of interest on Twitter and some discussion on Hacker News.
[... 452 words]Tracking PG&E outages by scraping to a git repo
PG&E have cut off power to several million people in northern California, supposedly as a precaution against wildfires.
[... 868 words]Weeknotes: Dogsheep
Having figured out my Stanford schedule, this week I started getting back into the habit of writing some code.
[... 1,367 words]Weeknotes: first week of Stanford classes
One of the benefits of the JSK fellowship is that I can take classes and lectures at Stanford, on a somewhat ad-hoc basis (I don’t take exams or earn credits).
[... 544 words]Weeknotes: Design thinking for journalists, genome-to-sqlite, datasette-atom
I haven’t had much time for code this week: we’ve had a full five day workshop at JSK with Tran Ha (a JSK alumni) learning how to apply Design Thinking to our fellowship projects and generally to challenges facing journalism.
[... 870 words]Weeknotes: ONA19, twitter-to-sqlite, datasette-rure
I’ve decided to start writing weeknotes for the duration of my JSK fellowship. Here goes!
[... 919 words]My JSK Fellowship: Building an open source ecosystem of tools for data journalism
I started a new chapter of my career last week: I began a year long fellowship with the John S. Knight Journalism Fellowships program at Stanford.
[... 876 words]Single sign-on against GitHub using ASGI middleware
I released Datasette 0.29 last weekend, the first version of Datasette to be built on top of ASGI (discussed previously in Porting Datasette to ASGI, and Turtles all the way down).
[... 1,612 words]Porting Datasette to ASGI, and Turtles all the way down
This evening I finally closed a Datasette issue that I opened more than 13 months ago: #272: Port Datasette to ASGI. A few notes on why this is such an important step for the project.
[... 1,082 words]Datasette 0.28—and why master should always be releasable
It’s been quite a while since the last substantial release of Datasette. Datasette 0.27 came out all the way back in January.
[... 1,326 words]Running Datasette on Glitch
Update 28th May 2025: Sadly Glitch is shutting down user project hosting, so this tutorial is no longer relevant.
[... 1,020 words]Generating a commit log for San Francisco’s official list of trees
San Francisco has a neat open data portal (as do an increasingly large number of cities these days). For a few years my favourite file on there has been Street Tree List, a list of all 190,000 trees in the city maintained by the Department of Public Works.
[... 1,051 words]I commissioned an oil painting of Barbra Streisand’s cloned dogs
Last year, Barbra Streisand cloned her dog, Sammie.
[... 517 words]sqlite-utils: a Python library and CLI tool for building SQLite databases
sqlite-utils is a combination Python library and command-line tool I’ve been building over the past six months which aims to make creating new SQLite databases as quick and easy as possible.
[... 1,237 words]Exploring search relevance algorithms with SQLite
SQLite isn’t just a fast, high quality embedded database: it also incorporates a powerful full-text search engine in the form of the FTS4 and FTS5 extensions. You’ve probably used these a bunch of times already: many iOS, Android and desktop applications use SQLite under-the-hood and use it to implement their built-in search.
[... 1,390 words]Fast Autocomplete Search for Your Website
Every website deserves a great search engine—but building a search engine can be a lot of work, and hosting it can quickly get expensive.
[... 4,159 words]Building smaller Python Docker images
Changes are afoot at Zeit Now, my preferred hosting provider for the past year (see previous posts). They have announced Now 2.0, an intriguing new approach to providing auto-scaling immutable deployments. It’s built on top of lambdas, and comes with a whole host of new constraints: code needs to fit into a 5MB bundle for example (though it looks like this restriction will soon be relaxed a little—update November 19th you can now bump this up to 50MB).
[... 1,872 words]Automatically playing science communication games with transfer learning and fastai
This weekend was the 9th annual Science Hack Day San Francisco, which was also the 100th Science Hack Day held worldwide.
[... 1,174 words]How to Instantly Publish Data to the Internet with Datasette
I spoke about my Datasette project at PyBay in August and they’ve just posted the video of my talk.
[... 58 words]How I moderated the State of Django panel at DjangoCon US.
On Wednesday last week I moderated the State of Django panel as the closing session for DjangoCon US 2018.
[... 1,210 words]The interesting ideas in Datasette
Datasette (previously) is my open source tool for exploring and publishing structured data. There are a lot of ideas embedded in Datasette. I realized that I haven’t put many of them into writing.
[... 2,857 words]Letterboxing on Lundy
Last week Natalie and I spent a delightful two days with our friends Hannah and Adam on the beautiful island of Lundy in the Bristol Channel, 12 miles off the coast of North Devon.
[... 1,461 words]The subset of reStructuredText worth committing to memory
reStructuredText is the standard for documentation in the Python world.
[... 1,186 words]How to Instantly Publish Data to the Internet with Datasette
I presented a session about Datasette at the PyBay 2018 conference in San Francisco. I talked about the project itself and demonstrated ways of creating and publishing databases using csvs-to-sqlite, Datasette Publish and my new sqlite-utils library.
[... 2,043 words]