Simon Willison’s Weblog

Entries in Apr

Filters: Type: entry × Month: Apr ×


Weeknotes: Archiving coronavirus.data.gov.uk, custom pages and directory configuration in Datasette, photos-to-sqlite

I mainly made progress on three projects this week: Datasette, photos-to-sqlite and a cleaner way of archiving data to a git repository.

[... 1132 words]

Weeknotes: Datasette 0.40, various projects, Dogsheep photos

A new release of Datasette, two new projects and progress towards a Dogsheep photos solution.

[... 826 words]

Using a self-rewriting README powered by GitHub Actions to track TILs

I’ve started tracking TILs—Today I Learneds—inspired by this five-year-and-counting collection by Josh Branchaud on GitHub (found via Hacker News). I’m keeping mine in GitHub too, and using GitHub Actions to automatically generate an index page README in the repository and a SQLite-backed search engine.

[... 1100 words]

Weeknotes: Hacking on 23 different projects

I wrote a lot of code this week: 184 commits over 23 repositories! I’ve also started falling for Zeit Now v2, having found workarounds for some of my biggest problems with it.

[... 901 words]

datasette-clone

I released a fun little Datasette utility today: datasette-clone.

[... 369 words]

Goodbye Zeit Now v1, hello datasette-publish-now—and talking to myself in GitHub issues

This week I’ve been mostly dealing with the finally announced shutdown of Zeit Now v1. And having long-winded conversations with myself in GitHub issues.

[... 2049 words]

Weeknotes: Covid-19, First Python Notebook, more Dogsheep, Tailscale

My covid-19.datasettes.com project publishes information on COVID-19 cases around the world. The project started out using data from Johns Hopkins CSSE, but last week the New York Times started publishing high quality USA county- and state-level daily numbers to their own repository. Here’s the change that added the NY Times data.

[... 993 words]

Running Datasette on Glitch

The worst part of any software project is setting up a development environment. It’s by far the biggest barrier for anyone trying to get started learning to code. I’ve been a developer for more than twenty years and I still feel the pain any time I want to do something new.

[... 998 words]

Exploring the UK Register of Members Interests with SQL and Datasette

Ever wondered which UK Members of Parliament get gifted the most helicopter rides? How about which MPs have been given Christmas hampers by the Sultan of Brunei? (David Cameron, William Hague and Michael Howard apparently). Here’s how to dig through the Register of Members Interests using SQL and Datasette.

[... 1167 words]

Datasette plugins, and building a clustered map visualization

Datasette now supports plugins!

[... 751 words]

Looking for a modern day wiki for a group project

GitHub offers a wiki for each repository. It’s free for public projects (no need to upload any code, just create a repo and ignore the other tabs) of you can pay $7 monthly for private ones.

[... 87 words]

What is the best place to post a call for papers in the computer science community?

Our service Lanyrd has a feature to help keep track of conference CFPs. We tend to have more commercial conferences than academic ones, but those topics are definitely relevant to our core audience. For example, our Cloud conferences and events page shows six upcoming CFPs at the moment.

[... 107 words]

What are some interesting and unusual tactics to draw people to your booth at a large conference or convention?

Make sure you have people on the stand who know their stuff—ideally people who built the thing you are promoting. Make it worthwhile for people to come and talk to you. Don’t have booth babes who are there for their looks rather than their expertise.

[... 129 words]

Which should I learn: Python or PHP?

Python will teach you more about programming than PHP—and you’ll be able to learn PHP easily if you learn Python first.

[... 36 words]

What could be the reason my login with Twitter stopped work in web site?

Yes, there have been breaking changes recently. You need to make sure you are running on Twitter’s API v1.1: https://dev.twitter.com/docs/api...

[... 41 words]

How should I start learning Python?

I’d start by going through the official Python tutorial. It’s really good: The Python Tutorial

[... 28 words]

What are the best books/tutorials to begin learning about memcached?

There isn’t really enough of memcached to justify a whole book—it’s a pretty straight-forward API.

[... 100 words]

What’s the best material on scalability?

Cal Henderson’s book Building Scalable Websites offers a good grounding.

[... 32 words]

What are the simple ways  to gain more self confidence in public speaking?

Be confident that you understand your subject matter. Practice, out loud, in private, as many times as possible. Have notes in your pocket so you know you can fall back on them if you really need to (you probably won’t, but I find knowing they are there helps with nerves). Then get as much speaking experience as you can. BarCamps / Unconferences are a useful place to get started.

[... 104 words]

How should one maximize their professional conference experience?

Take a look at How to maximize the conference experience as an attendee?

[... 28 words]

What is the best design conference in the midwest?

If you’re looking for a general UX/web design event, the An Event Apart series are excellent—they tour various American cities including Chicago from August 27-29. http://aneventapart.com/2012/chi...

[... 140 words]

Is there a feed or otherwise centralized list of calls for proposals for technical conferences?

We have a crowdsourced index of calls for proposals from (mostly technical) conferences here: http://lanyrd.com/calls/—you can also subscribe to the RSS feed for individual topics, e.g. if you want to hear about PHP conference CFPs you can subscribe to the feed linked to from this page: http://lanyrd.com/topics/php/calls/

[... 72 words]

Which database engine does Lanyrd use?

We’re on MySQL (InnoDB), using Amazon’s RDS. We have enough to worry about without tuning and configuring our own database server.

[... 33 words]

Comprehensive notes from my three hour Redis tutorial

Last week I presented two talks at the inaugural NoSQL Europe conference in London. The first was presented with Matthew Wall and covered the ways in which we have been exploring NoSQL at the Guardian. The second was a three hour workshop on Redis, my favourite piece of software to have the NoSQL label applied to it.

[... 263 words]

WildlifeNearYou talk at £5 app, and being Wired (not Tired)

Two quick updates about WildlifeNearYou. First up, I gave a talk about the site at £5 app, my favourite Brighton evening event which celebrates side projects and the joy of Making Stuff. I talked about the site’s genesis on a fort, crowdsourcing photo ratings, how we use Freebase and DBpedia and how integrating with Flickr’s machine tags gave us a powerful location API for free. Here’s the video of the talk, courtesy of Ian Oszvald:

[... 171 words]

rev=canonical bookmarklet and designing shorter URLs

I’ve watched the proliferation of URL shortening services over the past year with a certain amount of dismay. I care about the health of the web and try to ensure that URLs I am responsible will last for as long as possible, and I think it’s very unlikely that all of these new services will still be around in twenty years time. Last month I suggested that the Internet Archive start mirroring redirect databases, and last week I was pleased to hear that Archiveteam, a different organisation, had already started crawling.

[... 920 words]

Turn your Django application in to an OpenID consumer

I’ve just put the finishing touches on the first releasable version of django_openidconsumer, a package that makes it easy to add OpenID consumer support to any Django application.

[... 229 words]

Extending a WiFi network with two Macs and a FireWire cable

Last night’s Oxford Geek Night went really well, despite more than the usual flurry of problems. It’s definitely true that the more geeks there are in a room the less likely it is that the projector will work! Thankfully we got everything up and running in time for the talks to start, although it was a pretty close call.

[... 595 words]

San Diego, tell me more

You should totally go to Horton Plaza. It’s a shopping mall, but it’s also basically one big architectural joke. Everything is at funny angles, and stairs and ramps never take you where they think you will. I usually have no interest in visiting malls at all but I totally loved it.

[... 64 words]

Exciting stuff in Python 2.5

Python 2.5 alpha 1 is out, and as usual the What’s New in Python 2.5 document provides a pleasant overview of the new features. There are some real treats in there. While I’m hoping that the syntax for conditional expressions will grow on me, I’m looking forward to Partial function application becoming a common Python idiom. Relative imports are going to make Django applications a lot easier to redistribute, and I can’t wait to see all the crazy hacks that result from the introduction of coroutines.

[... 291 words]