Simon Willison’s Weblog

Entries in Mar

Filters: Type: entry × Month: Mar ×


Weeknotes: SpatiaLite 5, Datasette on Azure, more CDC vaccination history

This week I got SpatiaLite 5 working in the Datasette Docker image, improved the CDC vaccination history git scraper, figured out Datasette on Azure and we closed on a new home!

[... 986 words]

Weeknotes: django-sql-dashboard widgets

A few small releases this week, for django-sql-dashboard, datasette-auth-passwords and datasette-publish-vercel.

[... 1025 words]

Weeknotes: tableau-to-sqlite, django-sql-dashboard

This week I started a limited production run of my new backend for Vaccinate CA calling, built a tableau-to-sqlite import tool and started working on a subset of Datasette for PostgreSQL and Django called django-sql-dashboard.

[... 792 words]

Weeknotes: Datasette and Git scraping at NICAR, VaccinateCA

This week I virtually attended the NICAR data journalism conference and made a ton of progress on the Django backend for VaccinateCA (see last week).

[... 773 words]

Git scraping, the five minute lightning talk

I prepared a lightning talk about Git scraping for the NICAR 2021 data journalism conference. In the talk I explain the idea of running scheduled scrapers in GitHub Actions, show some examples and then live code a new scraper for the CDC’s vaccination data using the GitHub web interface. Here’s the video.

[... 289 words]

Weeknotes: Datasette 0.39 and many other projects

This week’s theme: Well, I’m not going anywhere. So a ton of progress to report on various projects.

[... 806 words]

Weeknotes: this week was absurd

As of this morning, San Francisco is in a legally mandated shelter-in-place. I can hardly remember what life was like seven days ago. It’s been a very long, very crazy week. This was not a great week for getting stuff done.

[... 246 words]

Weeknotes: COVID-19 numbers in Datasette

COVID-19, the disease caused by the novel coronavirus, gets more terrifying every day. Johns Hopkins Center for Systems Science and Engineering (CSSE) have been collating data about the spread of the disease and publishing it as CSV files on GitHub.

[... 644 words]

datasette-search-all: a new plugin for searching multiple Datasette tables at once

I just released a new plugin for Datasette, and it’s pretty fun. datasette-search-all is a plugin written mostly in JavaScript that executes the same search query against every searchable table in every database connected to your Datasette instance.

[... 819 words]

Weeknotes: datasette-ics, datasette-upload-csvs, datasette-configure-fts, asgi-csrf

I’ve been preparing for the NICAR 2020 Data Journalism conference this week which has lead me into a flurry of activity across a plethora of different projects and plugins.

[... 834 words]

Generating a commit log for San Francisco’s official list of trees

San Francisco has a neat open data portal (as do an increasingly large number of cities these days). For a few years my favourite file on there has been Street Tree List, a list of all 190,000 trees in the city maintained by the Department of Public Works.

[... 1051 words]

I commissioned an oil painting of Barbra Streisand’s cloned dogs

Two dogs in a stroller looking at a gravestone, as an oil painting
Two identical puffs of white fur, gazing at the tombstone of the dog they are

[... 517 words]

Building a combined stream of recent additions using the Django ORM

I’m a big believer in the importance of a “recent additions” feed. Any time you’re building an application that involves users adding and editing records it’s useful to have a page somewhere that shows the most recent objects that have been created across multiple different types of data.

[... 1647 words]

How do I receive automatic updates from a Facebook group by email?

Facebook’s API does provide a feed of recent posts to a group: https://developers.facebook.com/docs/graph-api/reference/v2.8/group/feed

[... 85 words]

Where do you like to eat in London?

I adore Ya Hala next to Paddington station—Lebanese food (so lots of veggie options), open until midnight, really good tea as well. It’s my first stop in London whenever I’m visiting, since you can get directly there from Hearhrow on the Heathrow Express train.

[... 61 words]

Work process vs technology

Do you have a plan for what happens if you lose your hard drive, or someone steals it? I understand your need for offline access, but personally I’m terrified of losing my laptop to the point that I use cloud backup services (Dropbox, but I’ve used and liked Backblaze in the past) to make absolutely sure that I don’t lose any data should my laptop get lost or stolen.

[... 81 words]

We applied to Y Combinator a few weeks ago and submitted a demo. We made significant progress since then and have a much better demo now. Is it possible to replace the demo? Is it advisable?

It’s fine to replace the demo, and definitely a good idea if your new demo is better. That said, don’t expect that anyone who has viewed the old demo will check out the new one. As a result the sooner you get the new demo in place the better.

[... 91 words]

Have you ever experienced a boost in productivity by switching to a different programming language?

Switching from PHP to Python (over a decade ago now) dramatically improved my productivity as a programmer.

[... 38 words]

How does one decide which Javascript framework (e.g. Node, Backbone, Angular) to use on any given project?

If you are just learning JavaScript, I suggest trying to work without any frameworks or libraries at all. Starting with something like Angular will make it much harder for you to learn the core language and browser APIs.

[... 137 words]

What are some tips for improving public speaking skills quickly?

Practice your talk, out loud, in private, as many times as possible before you deliver it. There’s no better way of ensuring you know your material and that you can deliver it at a sensible pace without freezing up.

[... 127 words]

Are there any YC startups based around open source software?

There have been a few.

[... 99 words]

Does anyone ever had the experience where they could carry on a conversation with someone very well, but stutter and go blank when public speaking (eg. informal introduction) in front of a crowd?

This is extremely common, and will certainly go away with experience... but being well prepared is never a bad idea with respect to public speaking! Even for informal introductions it’s worth figuring out what you’ll say, writing it down, rehearsing it out loud several times in private and memorising it.

[... 132 words]

What are some ways to organize conferences at a very low cost?

Find a venue sponsor: venues are the biggest expense for most events, and if you can find a company with their own conference-suitable facilities who will provide the space for free you’ve just massively slashed the cost of the event. Most BarCamps (at least in London) work on this basis.

[... 158 words]

Why do academic conferences cost so much?

They’re usually a whole lot cheaper than industry conferences, where tickets can cost more than a thousand dollars.

[... 92 words]

What are some ways to get a job from attending a conference?

Talk to people. Ask if they are hiring—if they aren’t, ask if they know anyone at the conference who is hiring. Make sure you have lots of business cards. Get other people’s business cards. Follow up afterwards. Connect with people on LinkedIn.

[... 111 words]

What are some hidden features of Python?

Generators and Iterators are pretty amazing. These two tutorials will really open your eyes as to how powerful they can be:

[... 63 words]

Where can I find out about upcoming conferences, conventions, and trade shows in design, publishing, and elearning?

Our site, Lanyrd, is a crowdsourced directory of conferences and professional events. We’re extremely strong in areas such as Mobile, Web Design and User Experience, but we also have listings for Publishing and E-Learning. Try these pages:

[... 162 words]

How do I get started with node.js?

You don’t run Node.js in a script tag on your website. Node is a technology for writing servers in JavaScript—you can think of it as a replacement/alternative for PHP, Python, Ruby, ASP.NET etc.

[... 58 words]

How do you get invited to speak at high profile events?

SXSW has an open call for proposals—you’ll need to submit a suggestion to the panel picker and encourage people to vote for you.

[... 123 words]

How does one create innovation in a business conference? What’s new and different in this area?

Two formats worth investigating are unconferences and lightning talks.

[... 155 words]