Simon Willison’s Weblog

26th June 2020 - Link Blog

Reducing search indexing latency to one second. Really detailed dive into the nuts and bolts of Twitter’s latest iteration of search indexing technology, including a great explanation of skip lists.

Posted 26th June 2020 at 5:06 pm

Recent articles

sqlite-utils 4.0, now with database schema migrations - 7th July 2026
sqlite-utils 4.0rc2, mostly written by Claude Fable (for about $149.25) - 5th July 2026
Have your agent record video demos of its work with shot-scraper video - 30th June 2026

This is a link post by Simon Willison, posted on 26th June 2020.

data-structures 6 lucene 12 scaling 148 search 93 twitter 161

Monthly briefing

Sponsor me for $10/month and get a curated email digest of the month's most important LLM developments.

Pay me to send you less!

Sponsor & subscribe