Simon Willison’s Weblog

Bluesky WebSocket Firehose. Very quick (10 seconds of Claude hacking) prototype of a web page that attaches to the public Bluesky WebSocket firehose and displays the results directly in your browser.

Here's the code - there's very little to it, it's basically opening a connection to wss://jetstream2.us-east.bsky.network/subscribe?wantedCollections=app.bsky.feed.post and logging out the results to a <textarea readonly> element.

Bluesky's Jetstream isn't their main atproto firehose - that's a more complicated protocol involving CBOR data and CAR files. Jetstream is a new Go proxy (source code here) that provides a subset of that firehose over WebSocket.

Jetstream was built by Bluesky developer Jaz, initially as a side-project, in response to the surge of traffic they received back in September when Brazil banned Twitter. See Jetstream: Shrinking the AT Proto Firehose by >99% for their description of the project when it first launched.

The API scene growing around Bluesky is really exciting right now. Twitter's API is so expensive it may as well not exist, and Mastodon's community have pushed back against many potential uses of the Mastodon API as incompatible with that community's value system.

Hacking on Bluesky feels reminiscent of the massive diversity of innovation we saw around Twitter back in the late 2000s and early 2010s.

Here's a much more fun Bluesky demo by Theo Sanderson: firehose3d.theo.io (source code here) which displays the firehose from that same WebSocket endpoint in the style of a Windows XP screensaver.

# 20th November 2024, 4:05 am / websockets, bluesky, twitter, apis, mastodon

Ralph Sheldon’s Portrait of Henry VIII Reidentified (via) Here's a delightful two part story on art historian Adam Busiakiewicz's blog. Adam was browsing Twitter when he spotted this tweet by Tim Cox, Lord Lieutenant of Warwickshire, celebrating a reception.

He noticed a curve-framed painting mounted on a wall in the top left of the photo:

Adam had previously researched a similar painting while working at Sotheby's:

Seeing this round topped portrait immediately reminded me of a famous set of likenesses commissioned by the local politician and tapestry maker Ralph Sheldon (c. 1537--1613) for his home Weston House, Warwickshire, during the 1590s. Consisting of twenty-two portraits, mostly images of Kings, Queens and significant contemporary international figures, only a handful are known today.

Adam contacted Warwickshire County Council and was invited to Shire Hall. In his follow-up post he describes his first-hand observations from the visit.

It turns out the painting really was one of those 22 portraits made for tapestry maker Ralph Sheldon in the 1590s, long thought lost. The discovery has now made international news:

• BBC News: Missing Henry VIII portrait found after random X post
• Smithsonian Magazine: Art Historian Discovers Long-Lost Portrait of Henry VIII in Background of Social Media Post

# 30th July 2024, 11:12 pm / twitter, art, history

My Twitter thread figuring out the AI features in Microsoft’s Recall. I posed this question on Twitter about why Microsoft Recall (previously) is being described as "AI":

Is it just that the OCR uses a machine learning model, or are there other AI components in the mix here?

I learned that Recall works by taking full desktop screenshots and then applying both OCR and some sort of CLIP-style embeddings model to their content. Both the OCRd text and the vector embeddings are stored in SQLite databases (schema here, thanks Daniel Feldman) which can then be used to search your past computer activity both by text but also by semantic vision terms - "blue dress" to find blue dresses in screenshots, for example. The si_diskann_graph table names hint at Microsoft's DiskANN vector indexing library

A Microsoft engineer confirmed on Hacker News that Recall uses on-disk vector databases to provide local semantic search for both text and images, and that they aren't using Microsoft's Phi-3 or Phi-3 Vision models. As far as I can tell there's no LLM used by the Recall system at all at the moment, just embeddings.

# 5th June 2024, 10:39 pm / twitter, ai, embeddings, microsoft, sqlite, recall

Commit: Add a shared credentials relationship from twitter.com to x.com (via) A commit to shared-credentials.json in Apple's password-manager-resources repository. Commit message: "Pour one out."

# 17th May 2024, 8:04 pm / twitter

“Link In Bio” is a slow knife (via) Anil Dash writing in 2019 about how Instagram’s “link in bio” thing (where users cannot post links to things in Instagram posts or comments, just a single link field in their bio) is harmful for linking on the web.

Today it’s even worse. TikTok has the same culture, and LinkedIn and Twitter both algorithmically de-boost anything with a URL in it, encouraging users to share screenshots (often unsourced) rather than linking to content and reducing their distribution.

It’s gross.

# 12th May 2024, 2:15 pm / linkedin, social-media, links, twitter, anil-dash, tiktok

Latest Twitter search results for “as an AI language model” (via) Searching for “as an AI language model” on Twitter reveals hundreds of bot accounts which are clearly being driven by GPT models and have been asked to generate content which occasionally trips the ethical guidelines trained into the OpenAI models.

If Twitter still had an affordable search API someone could do some incredible disinformation research on top of this, looking at which accounts are implicated, what kinds of things they are tweeting about, who they follow and retweet and so-on.

# 17th April 2023, 2:28 pm / twitter, ethics, generative-ai, openai, misinformation, ai

Welcome to hell, Elon (via) If you only read one thing about the Elon acquisition of Twitter make it this, by Nilay Patel. Outstanding insights into what it actually takes to to run a commercial social media service.

# 28th October 2022, 3:16 pm / social-media, twitter, moderation, nilay-patel

Twitter pranksters derail GPT-3 bot with newly discovered “prompt injection” hack. I’m quoted in this Ars Technica article about prompt injection and the Remoteli.io Twitter bot.

# 16th September 2022, 6:33 pm / twitter, prompt-engineering, prompt-injection, gpt-3, security, openai, generative-ai, llms

@newshomepages (via) Ben Welsh used my shot-scraper tool and GitHub Actions to launch a Twitter bot which tweets screenshots of newspaper homepages on a scheduled basis. Ben says: “The tech is so easy, I was able to pull it off in a couple hours at zero cost. A decade ago I ran a similar project using the cloud resources of the day. [...] It costs thousands of dollars and the screenshots were of much lower quality. Incredible progress!”

# 12th March 2022, 7:21 pm / twitter, shot-scraper, github-actions, playwright, ben-welsh

A museum bot (via) Shawn Graham built a Twitter bot, using R, which tweets out random items from the collection at the Canadian Science and Technology Museum—using a Datasette instance that he’s running based on a CSV export of their collections data.

# 5th May 2021, 7:09 pm / twitter, datasette, museums

Reducing search indexing latency to one second. Really detailed dive into the nuts and bolts of Twitter’s latest iteration of search indexing technology, including a great explanation of skip lists.

# 26th June 2020, 5:06 pm / twitter, data-structures, scaling, search, lucene

twitter-to-sqlite 0.6, with track and follow. I shipped a new release of my twitter-to-sqlite command-line tool this evening. It now includes experimental features for subscribing to the Twitter streaming API: you can track keywords or follow users and matching Tweets will be written to a SQLite database in real-time as they come in through the API. Since Datasette supports mutable databases now you can run Datasette against the database and run queries against the tweets as they are inserted into the tables.

# 6th October 2019, 4:54 am / realtime, projects, twitter, dogsheep

My Twitter thread collecting behind the scenes content about Spider-Man: Into the Spider-Verse. I absolutely loved Spider-Verse, and I’ve been delighted to discover that many of the artists who created the movie are active on Twitter and have been posting all kinds of fascinating material about their creative process. I’ve been collecting examples in this Twitter thread for a couple of months now. They definitely deserved that Oscar.

# 25th February 2019, 2:57 pm / movies, twitter, spiderverse

Twitter conversation about long-term pre-paid archival storage. I kicked off a conversation on Twitter yesterday about long-time archival storage of web content: “Anyone know of a web hosting provider where I can pay a lump sum of money to host a file at a reliable URL essentially forever? Is this even remotely feasible?”. The thread is really interesting—this is definitely an unsolved problem, and it’s clear that the challenge is more organizational (how do you create an entity that can keep this kind of promise—does it need to be some kind of foundation or trust?) than technical.

# 5th November 2018, 6:50 pm / archiving, twitter

python-twitter/get_access_token.py. Creating an OAuth token for accessing a specific Twitter account is way harder than it needs to be. I was about to write my own command-line script for doing this using PIN-based authentication (where you pop open a browser showing the Twitter login flow, then get a PIN number at the end which you paste back into your script) when I discovered that the python-twitter library already ships with a script to do exactly that. Just run “python get_access_token.py”, paste in your app’s consumer key and secret, follow a link, enter the resulting PIN and the script will spit out the consumer_key / consumer_secret / access_token_key / access_token_secret combo you need to start using the Twitter API.

# 28th October 2018, 5:25 pm / twitter, oauth