13 items tagged “paul-ford”
2020
I get asked a lot about learning to code. Sure, if you can. It's fun. But the real action, the crux of things, is there in the database. Grab a tiny, free database like SQLite. Import a few million rows of data. Make them searchable. It's one of the most soothing activities known to humankind, taking big piles of messy data and massaging them into the rigid structure required of a relational database. It's true power.
Fun with binary data and SQLite
This week I’ve been mainly experimenting with binary data storage in SQLite. sqlite-utils can now insert data from binary files, and datasette-media can serve content over HTTP that originated as binary BLOBs in a database file.
[... 957 words]2019
Why I (Still) Love Tech: In Defense of a Difficult Industry (via) If you only read one longform piece this week, make it this one. Utterly delightful prose and a bunch of different messages that resonated with me deeply.
Imagine if you were really into the group Swervedriver in the mid-’90s but by 2019 someone was on CNBC telling you that Swervedriver represented, I don’t know, 10 percent of global economic growth, outpacing returns in oil and lumber. That’s the tech industry.
Usable Data (via) A Paul Ford essay from February 2016 in which he advocates for SQLite as the ideal format for sharing interesting data. I don’t know how I missed this one—it predates Datasette, but it perfectly captures the benefits that I’m trying to expose with the project. “In my dream universe, there would be a massive searchable torrent site filled with open, explorable data sets, in SQLite format, some with full text search indexes already in place.”
2018
Consider Bitcoin a grand middle finger. It’s a prank, almost a parody of the global financial system, that turned into a bubble. “You plutocrats of Davos may think you control the global money supply,” the pranksters seem to say. “But humans will make an economy out of anything. Even this!”
How I made a Who’s On First subset database. Inspired by Paul Ford on Twitter, I tried out a new trick with SQLite: connect to a database containing JSON, attach a brand new empty database file using “attach database”, then populate it using INSERT INTO ... SELECT plus the json_extract() function to extract out a subset of the JSON properties into a new table in the new database.
[On SQLite] The JSON interface is like, "we save the text and when you retrieve it we parse the JSON at several hundred MB/s and let you do path queries against it please stop overthinking it, this is filing cabinet."
2017
Datasette: Ability to customize presentation of specific columns in HTML view.
Still a work in progress, but Datasette master now allows you to inject links to one or more additional CSS and JavaScript resources (optionally with SRI hashes) which will be included on every page. Each template also
now provides CSS classes on the body element derived from the current database and table names to provide hooks for custom styling. Next up: custom template support.
If you’re a public data provider—and many large NGOs, government organizations, cultural organizations, historical archives, media organizations, medical orgs, and academic institutions are exactly that—you can publish gigabytes of data, and make it available as an API, and make it easy to browse on the web, too, with extremely low effort. Put it into SQLite, point this little guy at it, and you’ve just radically increased the accessibility and utility of your data. Because messing around in SQL from a web browser is orders of magnitude more immediately useful than downloading a CSV, processing it, and figuring out what comes next.
2008
Learning to Fear the Semantic Web. Paul Ford raises the liability issue with regards to building sites around other people’s metadata, pointing out that OpenCalais is owned by Thomson Reuters who have a bad track record with regards to intellectual property lawsuits elsewhere in the organisation.
2007
Harper’s Magazine (via) The site with the best metadata on the Web just relaunched, with even MORE metadata.
If your average iPod weighs five ounces with packaging, then Apple has moved about 21,875,000 pounds of them, equivalent in weight to 1,325 full-grown male African elephants, 35 times as many as Hannibal's force.