Simon Willison’s Weblog

Subscribe

October 2020

77 posts: 7 entries, 16 links, 6 quotes, 48 beats

Oct. 11, 2020

Release xml-analyser 1.0 — Simple command line tool for quickly analysing the structure of an arbitrary XML file
Release evernote-to-sqlite 0.1 — Tools for converting Evernote content to SQLite

Oct. 12, 2020

Release evernote-to-sqlite 0.2 — Tools for converting Evernote content to SQLite

evernote-to-sqlite (via) The latest tool in my Dogsheep series of utilities for personal analytics: evernote-to-sqlite takes Evernote note exports en their ENEX XML format and loads them into a SQLite database. Embedded images are loaded into a BLOB column and the output of their cloud-based OCR system is added to a full-text search index. Notes have a latitude and longitude which means you can visualize your notes on a map using Datasette and datasette-cluster-map.

# 12:38 am / projects, sqlite, datasette, dogsheep

xml-analyser. In building evernote-to-sqlite I dusted off an ancient (2009) project I built that scans through an XML file and provides a summary of what elements are present in the document and how they relate to each other. I’ve now packaged it up as a CLI app and published it on PyPI.

# 12:41 am / cli, projects, xml

Release swarm-to-sqlite 0.3.2 — Create a SQLite database containing your checkin history from Foursquare Swarm

Oct. 14, 2020

None
TIL Upgrading Python Homebrew packages using pip — [VisiData 2.0](https://www.visidata.org/) came out today. I previously installed VisiData using Homebrew, but the VisiData tap has not yet been updated with the latest version.

Oct. 16, 2020

Release csv-diff 1.0 — Python CLI tool and library for diffing CSV and JSON files
Release sqlite-utils 2.22 — Python CLI utility and library for manipulating SQLite databases

Building an Evernote to SQLite exporter

Visit Building an Evernote to SQLite exporter

I’ve been using Evernote for over a decade, and I’ve long wanted to export my data from it so I can do interesting things with it.

[... 1,879 words]

None
TIL Explicit file encodings using click.File — I wanted to add a `--encoding` option to `sqlite-utils insert` which could be used to change the file encoding used to read the incoming CSV or TSV file - see [sqlite-utils #182](https://github.com/simonw/sqlite-utils/issues/182).

Weeknotes: evernote-to-sqlite, Datasette Weekly, scrapers, csv-diff, sqlite-utils

Visit Weeknotes: evernote-to-sqlite, Datasette Weekly, scrapers, csv-diff, sqlite-utils

This week I built evernote-to-sqlite (see Building an Evernote to SQLite exporter), launched the Datasette Weekly newsletter, worked on some scrapers and pushed out some small improvements to several other projects.

[... 827 words]

Oct. 17, 2020

Release healthkit-to-sqlite 1.0 — Convert an Apple Healthkit export zip to a SQLite database
Release dogsheep-beta 0.9 — Build a search index across content from multiple SQLite database tables and run faceted searches against it using Datasette

Oct. 18, 2020

Release sphinx-to-sqlite 0.1a — Create a SQLite database from Sphinx documentation

Oct. 19, 2020

Release github-to-sqlite 2.7 — Save data from GitHub to a SQLite database

The stampede of the affluent into grim-faced, highly competitive sports has been a tragicomedy of perverse incentives and social evolution in unequal times: a Darwinian parable of the mayhem that can ensue following the discovery of even a minor advantage. Like a peacock rendered nearly flightless by gaudy tail feathers, the overserved athlete is the product of a process that has become maladaptive, and is now harming the very blue-chip demographic it was supposed to help.

Ruth S. Barrett

# 3:09 pm / sports

Dogsheep: Personal analytics with Datasette. The second edition of my new Datasette Weekly newsletter, talks about Dogsheep, Dogsheep Beta, Datasette 1.0 and features datasette-cluster-map as the plugin of the week.

# 4:38 pm / datasette, dogsheep

Release datasette-cluster-map 0.13 — Datasette plugin that shows a map for any data with latitude/longitude columns

Oct. 20, 2020

Release datasette-json-preview 0.2 — Preview of new JSON default format for Datasette
Release datasette 0.51a0 — An open source multi-tool for exploring and publishing data

Oct. 21, 2020

Pikchr. Interesting new project from SQLite creator D. Richard Hipp. Pikchr is a new mini language for describing visual diagrams, designed to be embedded in Markdown documentation. It’s already enabled for the SQLite forum. Implementation is a no-dependencies C library and output is SVG.

# 4:02 pm / c, sqlite, svg, markdown, d-richard-hipp

Proof of concept: sqlite_utils magic for Jupyter (via) Tony Hirst has been experimenting with building a Jupyter “magic” that adds special syntax for using sqlite-utils to insert data and run queries. Query results come back as a Pandas DataFrame, which Jupyter then displays as a table.

# 5:26 pm / pandas, sqlite, tony-hirst, jupyter, sqlite-utils

Writing the code to sign data with a private key and verify it with a public key would have been easier to get correct than correctly invoking the JWT library. In fact, the iOS app (which gets this right) doesn’t use a JWT library at all, but manages to verify using a public key in fewer lines of code than the Android app takes to incorrectly use a JWT library!

James 'zofrex' Sanderson

# 9:34 pm / security, jwt

Oct. 22, 2020

Release pocket-to-sqlite 0.2.1 — Create a SQLite database containing data from your Pocket account
Release inaturalist-to-sqlite 0.2.1 — Create a SQLite database containing your observation history from iNaturalist
None
TIL Writing JavaScript that responds to media queries — I wanted to change the layout of [my blog](https://simonwillison.net/) on mobile screens such that the content from the "Elsewhere" right hand column combined with the main column in the correct order (issue [#165](https://github.com/simonw/simonwillisonblog/issues/165)). I couldn't find a way to do this in pure CSS without duplicating a bunch of content, so I decided to do it with JavaScript.
Release datasette-cluster-map 0.14 — Datasette plugin that shows a map for any data with latitude/longitude columns
Release sphinx-to-sqlite 0.1a1 — Create a SQLite database from Sphinx documentation

2020 » October

MTWTFSS
   1234
567891011
12131415161718
19202122232425
262728293031