Simon Willison’s Weblog

Entries

Filters: Type: entry ×


Running a load testing Go utility using Docker for Mac

I’m playing around with Zeit Now at the moment (see my previous entry) and decided to hit it with some traffic using Apache Bench. I got this SSL handshake error:

[... 818 words]

Using “import refs” to iteratively import data into Django

I’ve been writing a few scripts to backfill my blog with content I originally posted elsewhere. So far I’ve imported answers I posted on Quora (background), answers I posted on Ask MetaFilter and content I recovered from the Internet Archive.

[... 560 words]

Late night dining near Great American Music Hall

Tommy’s Joynt is a couple of blocks away and is a San Francisco institution—great comfort food, inexpensive, crammed with personality and open late.

[... 40 words]

Porting my blog to Python 3

This blog is now running on Python 3! Admittedly this is nearly nine years after the first release of Python 3.0, but it’s the first Python 3 project I’ve deployed myself so I’m pretty excited about it.

[... 883 words]

How to set up world-class continuous deployment using free hosted tools

I’m going to describe a way to put together a world-class continuous deployment infrastructure for your side-project without spending any money.

[... 1273 words]

Deploying an asynchronous Python microservice with Sanic and Zeit Now

Back in 2008 Natalie Downe and I deployed what today we would call a microservice: json-head, a tiny Google App Engine app that allowed you to make an HTTP head request against a URL and get back the HTTP headers as JSON. One of our initial use-scase for this was Natalie’s addSizes.js, an unobtrusive jQuery script that could annotate links to PDFs and other large files with their corresponding file size pulled from the Content-Length header. Another potential use-case is detecting broken links, since the API can be used to spot 404 status codes (as in this example).

[... 1361 words]

Changelogs to help understand the fires in the North Bay

The situation in the counties north of San Francisco is horrifying right now. I’ve repurposed some of the tools I built to for the Irma Response project last month to collect and track some data that might be of use to anyone trying to understand what’s happening up there. I’m sharing these now in the hope that they might prove useful.

[... 383 words]

Recovering missing content from the Internet Archive

When I restored my blog last weekend I used the most recent SQL backup of my blog’s database from back in 2010. I thought it had all of my content from before I started my 7 year hiatus, but in watching the 404 logs I started seeing the occasional hit to something that really should have been there but wasn’t. Turns out the SQL backup I was working from was missing some content.

[... 636 words]

Should I build my startup’s web-based product as if it’s going to one day be widely adopted and experience high-volume, or instead focus on quick delivery over scalability?

Absolutely the second: build for rapid learning, not for eventual scalability. The vast majority of startups fail, and the number one reason they fail is that they didn’t achieve product-market fit: they failed to build something that customers actually wanted.

[... 169 words]

Implementing faceted search with Django and PostgreSQL

I’ve added a faceted search engine to this blog, powered by PostgreSQL. It supports regular text search (proper search, not just SQL“like” queries), filter by tag, filter by date, filter by content type (entries vs blogmarks vs quotation) and any combination of the above. Some example searches:

[... 3049 words]

Running gunicorn behind nginx on Heroku for buffering and logging

Heroku’s default setup for Django uses the gunicorn application server. Each Heroku dyno can only run a limited number of gunicorn workers, which means a limited number of requests can be served in parallel (around 4 per dyno is a good rule of thumb).

[... 400 words]

Getting the blog back together

Getting this blog up and running again has turned out to be one of those side-projects that keeps threatening to fall down a rabbit hole.

[... 160 words]

Scraping hurricane Irma

The Irma Response project is a team of volunteers working together to make information available during and after the storm. There is a huge amount of information out there, on many different websites. The Irma API is an attempt to gather key information in one place, verify it and publish it in a reuseable way. It currently powers the irmashelters.org website.

[... 438 words]

Doc of docs

Here’s a low-tech, high-impact trick I recently learned at work that’s amazingly useful: create a doc-of-docs.

[... 215 words]

Software or project management technique for DIY home renovations?

We’ve been using a Trello board for this. Free, flexible, fun to use and has good mobile as well as desktop support. You can model “this task requires this task first” by linking to other cards in comments.

[... 54 words]

Free Afternoon in San Francisco with no car

My absolute favorite thing to do in SF is to walk up a hill. My favourite hill in the city is Corona Heights over in the Castro—it’s beautiful and the view is spectacular. It’s not part of the regular tourist circuit—you can think of it as a best-kept-secret of the city.

[... 132 words]

A thing to do in London that is special, unusual and fun

London has some of the best cocktail bars in the world. I’m a big fan of this top 50 list, which is updated every year based on an international panel of bartenders. It looks like the American Bar at the Savoy is back at the top for London now—it’s been serving cocktails for over 100 years including keeping the American cocktail tradition alive during prohibition. Sure, it’s expensive but your budget should cover a couple of drinks each (which for me is absolutely enough for an evening). Or you could hit a couple of the bars on that list and have a drink at each one

[... 168 words]

Podcasts for commuting to

The BBC world service Documentary podcast features “the best of BBC World Service documentaries and other factual programmes” and is absolutely spectacular. The quality is uniformly high.

[... 89 words]

Looking for a modern day wiki for a group project

GitHub offers a wiki for each repository. It’s free for public projects (no need to upload any code, just create a repo and ignore the other tabs) of you can pay $7 monthly for private ones.

[... 87 words]

How do I receive automatic updates from a Facebook group by email?

Facebook’s API does provide a feed of recent posts to a group: https://developers.facebook.com/docs/graph-api/reference/v2.8/group/feed

[... 85 words]

Where do you like to eat in London?

I adore Ya Hala next to Paddington station—Lebanese food (so lots of veggie options), open until midnight, really good tea as well. It’s my first stop in London whenever I’m visiting, since you can get directly there from Hearhrow on the Heathrow Express train.

[... 61 words]

Work process vs technology

Do you have a plan for what happens if you lose your hard drive, or someone steals it? I understand your need for offline access, but personally I’m terrified of losing my laptop to the point that I use cloud backup services (Dropbox, but I’ve used and liked Backblaze in the past) to make absolutely sure that I don’t lose any data should my laptop get lost or stolen.

[... 81 words]

Podcasts to love and fall asleep to

Not technically podcasts but the BBC’s radio output is still free to listen to from anywhere in the world (no ads!) and helps me get to sleep most nights. I just hit play in the browser on my phone.

[... 93 words]

Where to find well-balanced rationals behind Trump policies?

Reddit’s NeutralPolitics is a reasonably well moderated community that may be relevant for this.

[... 30 words]

San Francisco Recommendations: comics, fashion, food, and more

I’m not a huge comics person, but that said I strongly recommend a trip to Isotope Comics in Hayes Valley on a weekday. Go on a weekday when the owner is running the store. He is incredibly charismatic and positively vibrates with enthusiasm about comics. He also has an incredible collection of toilet seats lining the upper wall of the store, each one illustrated by a famous comic book artist. Make sure to ask him about them.

[... 122 words]

A guide to the guides to Paris

You can’t go wrong with Atlas Obscura. I haven’t used it in Paris (I’ve used it successfully in many other places though, including Mexico City) but from glancing over their listings it looks like there are some real gems there: http://www.atlasobscura.com/things-to-do/paris-france. I’ve been to the Catacombs and they are astonishing.

[... 78 words]

Help us plan a trip to the English countryside

The Mechanical Music Museum in Northleach in the Cotswolds is a hidden gem—definitely worth dropping in for their tour.

[... 37 words]

Simple, delicious eating during long weekend in the mountains

My wife has had great success with this vegan 5 ingredient sweet potato black bean chili recipe—its pretty straight forward to make, scales up extremely well (we’ve fed 15 people using two pots before) and tastes even better if you reheat it on subsequent days.

[... 64 words]

Wireless access points or mesh networking?

I recently acquired the Netgear Orbi based on this recommendation in the wirecutter and it provides excellent fast wifi across our whole apartment where our previous non-mesh router failed to do so (tested using fast.com)

[... 68 words]

How to tell my bosses I’m bored

The magic word here is “challenging”. Don’t say you are bored—say you do not find your current workload challenging enough. Any good manager should be ready to respond to that.

[... 46 words]