Simon Willison’s Weblog

Latest Twitter search results for “as an AI language model” (via) Searching for “as an AI language model” on Twitter reveals hundreds of bot accounts which are clearly being driven by GPT models and have been asked to generate content which occasionally trips the ethical guidelines trained into the OpenAI models.

If Twitter still had an affordable search API someone could do some incredible disinformation research on top of this, looking at which accounts are implicated, what kinds of things they are tweeting about, who they follow and retweet and so-on. # 17th April 2023, 2:28 pm

Welcome to hell, Elon (via) If you only read one thing about the Elon acquisition of Twitter make it this, by Nilay Patel. Outstanding insights into what it actually takes to to run a commercial social media service. # 28th October 2022, 3:16 pm

Twitter pranksters derail GPT-3 bot with newly discovered “prompt injection” hack. I’m quoted in this Ars Technica article about prompt injection and the Remoteli.io Twitter bot. # 16th September 2022, 6:33 pm

@newshomepages (via) Ben Welsh used my shot-scraper tool and GitHub Actions to launch a Twitter bot which tweets screenshots of newspaper homepages on a scheduled basis. Ben says: “The tech is so easy, I was able to pull it off in a couple hours at zero cost. A decade ago I ran a similar project using the cloud resources of the day. [...] It costs thousands of dollars and the screenshots were of much lower quality. Incredible progress!” # 12th March 2022, 7:21 pm

A museum bot (via) Shawn Graham built a Twitter bot, using R, which tweets out random items from the collection at the Canadian Science and Technology Museum—using a Datasette instance that he’s running based on a CSV export of their collections data. # 5th May 2021, 7:09 pm

Reducing search indexing latency to one second. Really detailed dive into the nuts and bolts of Twitter’s latest iteration of search indexing technology, including a great explanation of skip lists. # 26th June 2020, 5:06 pm

twitter-to-sqlite 0.6, with track and follow. I shipped a new release of my twitter-to-sqlite command-line tool this evening. It now includes experimental features for subscribing to the Twitter streaming API: you can track keywords or follow users and matching Tweets will be written to a SQLite database in real-time as they come in through the API. Since Datasette supports mutable databases now you can run Datasette against the database and run queries against the tweets as they are inserted into the tables. # 6th October 2019, 4:54 am

My Twitter thread collecting behind the scenes content about Spider-Man: Into the Spider-Verse. I absolutely loved Spider-Verse, and I’ve been delighted to discover that many of the artists who created the movie are active on Twitter and have been posting all kinds of fascinating material about their creative process. I’ve been collecting examples in this Twitter thread for a couple of months now. They definitely deserved that Oscar. # 25th February 2019, 2:57 pm

Twitter conversation about long-term pre-paid archival storage. I kicked off a conversation on Twitter yesterday about long-time archival storage of web content: “Anyone know of a web hosting provider where I can pay a lump sum of money to host a file at a reliable URL essentially forever? Is this even remotely feasible?”. The thread is really interesting—this is definitely an unsolved problem, and it’s clear that the challenge is more organizational (how do you create an entity that can keep this kind of promise—does it need to be some kind of foundation or trust?) than technical. # 5th November 2018, 6:50 pm

python-twitter/get_access_token.py. Creating an OAuth token for accessing a specific Twitter account is way harder than it needs to be. I was about to write my own command-line script for doing this using PIN-based authentication (where you pop open a browser showing the Twitter login flow, then get a PIN number at the end which you paste back into your script) when I discovered that the python-twitter library already ships with a script to do exactly that. Just run “python get_access_token.py”, paste in your app’s consumer key and secret, follow a link, enter the resulting PIN and the script will spit out the consumer_key / consumer_secret / access_token_key / access_token_secret combo you need to start using the Twitter API. # 28th October 2018, 5:25 pm

The science of the hashtag. Interesting analysis of how the #lessambitiousmovies hash tag took off thanks to retweets from a couple of key users with very creative followers. # 14th January 2011, 4:02 am

WebKit is Dropping HTML5 “popstate” Events (via) Really nasty bug with WebKit’s pushState support, discovered by Ben Cherry from Twitter. popState events get dropped if the user navigates while an outbound network request is in progress. # 30th October 2010, 7:41 am

RasterWeb: Lanyrd. Pete Prodoehl calls me out on Lanyrd’s integration with the Twitter auth API at the expense of OpenID. I’ve posted a comment with my justification—essentially, tying to Twitter’s ecosystem means I can actually implement the features I’ve been talking about building on top of OpenID for years, with far less engineering effort. # 31st August 2010, 8:49 pm

Lanyrd—the social conference directory. Nat and my new project, launched today and doing pretty well despite some early server hiccups. Sign in with Twitter to see conferences that your friends are speaking at, attending or tracking, then add your own events. We’re particularly keen on helping people build up a detailed profile of their previous talks, so adding older conferences is encouraged. # 31st August 2010, 7:41 pm

simplegeo’s python-oauth2. The Python OAuth library scene is frighteningly complicated at the moment. This seems to be the most actively maintained, and the readme includes working example code for talking to the Twitter API (including integration with Django auth). # 18th July 2010, 5:22 pm

Realtime Election Tweets. Jay Caines-Gooby’s realtime election tweet service, using Node.js, nginx and WebSocket with a Flash fallback. # 6th May 2010, 9:20 pm

twitter’s gizzard (via) Intriguing new open source project from Twitter. Gizzard is a sharding framework which provides a network service for partitioning data across arbitrary backend datastores, managing its own forwarding table to map key ranges to partitions and adding support for tree-based replication. # 11th April 2010, 9:39 pm

Twitter, reformatted. I wrote a Yahoo! Pipe to clean up Twitter’s RSS feeds—removing the username prefix and filtering out items that begin with “@” or “RT”.. # 18th March 2010, 1:10 am

twitter-text-conformance (via) This is a neat idea: Twitter have released open source libraries for parsing standard tweet syntax in Ruby and Java, but they’ve also released a set of YAML unit tests aimed at anyone who wants to implement the same parsing logic in other languages. # 6th February 2010, 3:39 pm

Follow a Museum day. It’s follow a museum on Twitter day. Useful directory of museum Twitter accounts around the world, organised by country. # 1st February 2010, 11:15 am

russell davies: datadecs. Personalised christmas decorations made from data from Twitter, Doppler, last.fm and Flickr. The Twitter snowman came from a 3D printer—the size of the head varies depending on your number of followers. Best of all though is the Flickr decoration which represents the apertures you’ve used over the past year. # 7th January 2010, 9:58 pm

Self-Proclaimed Social Media Gurus on Twitter Multiplying Like Rabbits (via) 15,740 of them, including 2,091 social media consultants, 807 social media experts, 445 social media gurus and 68 social media stars. # 4th January 2010, 1:49 am

Going evented with Node.js. Comprehensive Node.js tutorial—from basic principles to installation and writing a simple Twitter search command-line client application. # 17th November 2009, 1:09 pm

The Secret Identity of the Peep Show Tweeter. Like many others, I had assumed the Peep Show character accounts were “official”—especially when they started live-tweeting their thoughts in real time as the episodes were aired. Turns out it was actually a very clever fan. # 30th October 2009, 6:46 pm

Comcast: Twitter Has Changed The Culture Of Our Company. “Frank Eliason (@Comcastcares on Twitter) now has 11 people working under him simply to respond to information about Comcast being broadcast on Twitter.” # 21st October 2009, 9:56 am

Simon Willison (simonw) on Twitter. I just realised I’ve never actually linked to my Twitter account on my blog. This is mainly an experiment to see if doing so makes my follower count go up... # 29th September 2009, 9:49 pm

tr.im is “discontinuing service”. “However, all tr.im links will continue to redirect, and will do so until at least December 31, 2009.Your tweets with tr.im URLs in them will not be affected.”—these statements seem to contradict themselves. Will tr.im URLs in tweets stop working after December 31st or not? Any chance they could hand the domain over to the Internet Archive? At any rate, this is exactly why centralised URL shorteners are a harmful trend. # 10th August 2009, 11:06 am

The Anatomy Of The Twitter Attack. Long-winded explanation of the recent Twitter break-in, but you can scroll to the bottom for a numbered list summary. The attacker first broke in to a Twitter employee’s personal Gmail account by “recovering” it against an expired Hotmail account (which the attacker could hence register themselves). They gained access to more passwords by searching for e-mails from badly implemented sites that send you your password in the clear. # 20th July 2009, 12:55 am

Curating conversations. Chris Thorpe has open-sourced the Guardian’s moderated Twitter backchannel app, for displaying back channels at high profile (and hence high potential for abuse) events. It’s a Python application that runs on App Engine. # 16th July 2009, 7:34 pm

You should follow me on Twitter. Dustin Curtis did a simple A/B testing experiment on his blog and found that the text “you should follow me on Twitter” had the highest click-through rate—173% more effective than “I’m on Twitter”. # 15th July 2009, 10:43 am