Simon Willison’s Weblog

Going evented with Node.js. Comprehensive Node.js tutorial—from basic principles to installation and writing a simple Twitter search command-line client application. # 17th November 2009, 1:09 pm

The Secret Identity of the Peep Show Tweeter. Like many others, I had assumed the Peep Show character accounts were “official”—especially when they started live-tweeting their thoughts in real time as the episodes were aired. Turns out it was actually a very clever fan. # 30th October 2009, 6:46 pm

Comcast: Twitter Has Changed The Culture Of Our Company. “Frank Eliason (@Comcastcares on Twitter) now has 11 people working under him simply to respond to information about Comcast being broadcast on Twitter.” # 21st October 2009, 9:56 am

Simon Willison (simonw) on Twitter. I just realised I’ve never actually linked to my Twitter account on my blog. This is mainly an experiment to see if doing so makes my follower count go up... # 29th September 2009, 9:49 pm

tr.im is “discontinuing service”. “However, all tr.im links will continue to redirect, and will do so until at least December 31, 2009.Your tweets with tr.im URLs in them will not be affected.”—these statements seem to contradict themselves. Will tr.im URLs in tweets stop working after December 31st or not? Any chance they could hand the domain over to the Internet Archive? At any rate, this is exactly why centralised URL shorteners are a harmful trend. # 10th August 2009, 11:06 am

The Anatomy Of The Twitter Attack. Long-winded explanation of the recent Twitter break-in, but you can scroll to the bottom for a numbered list summary. The attacker first broke in to a Twitter employee’s personal Gmail account by “recovering” it against an expired Hotmail account (which the attacker could hence register themselves). They gained access to more passwords by searching for e-mails from badly implemented sites that send you your password in the clear. # 20th July 2009, 12:55 am

Curating conversations. Chris Thorpe has open-sourced the Guardian’s moderated Twitter backchannel app, for displaying back channels at high profile (and hence high potential for abuse) events. It’s a Python application that runs on App Engine. # 16th July 2009, 7:34 pm

You should follow me on Twitter. Dustin Curtis did a simple A/B testing experiment on his blog and found that the text “you should follow me on Twitter” had the highest click-through rate—173% more effective than “I’m on Twitter”. # 15th July 2009, 10:43 am

YQL: INSERT INTO internet. insert into twitter.status (status,username,password) values (“Playing with INSERT, UPDATE and DELETE in YQL”, “twitterusername”,“twitterpassword”) # 8th July 2009, 8:19 pm

Up and running with Cassandra. Twitter are beginning to use Cassandra, the open source branch of Facebook’s BigTable-like non-relational database. Evan Weaver explains how to get started with it, but warns that it’s not yet a good idea to trust data to it without having a full backup in an unrelated storage engine. # 7th July 2009, 11:18 am

cache-money. A “write-through caching library for ActiveRecord”, maintained by Nick Kallen from Twitter. Queries hit memcached first, and caches are automatically kept up-to-date when objects are created, updated and deleted. Only some queries are supported—joins and comparisons won’t hit the cache, for example. # 28th June 2009, 3:17 pm

Twitter, an Evolving Architecture. The most detailed write-up of Twitter’s current architecture I’ve seen, explaining the four layers of cache (all memcached) used by the Twitter API. # 28th June 2009, 3:09 pm

C64 Twitter client. Awesome. # 17th June 2009, 9:14 am

The Twitpocalypse is Near: Will Your Twitter Client Survive? Twitter tweet IDs will shortly tick over past the maximum signed 32 bit integer, potentially breaking applications. I learnt this lesson when the same thing happened to Flickr photo IDs: never store numeric IDs from external systems as integers, always use strings. # 9th June 2009, 10:52 am

Muck Rack: Links posted by Guardian Journalists on Twitter. I’m rather impressed by the Sawhorse Media collection of Twitter aggregation sites (Muck Rack aggregates journalists)—a simple idea very well executed. Here’s a nice example—this page shows links posted to Twitter by known Guardian journalists, but goes a step further and scrapes in the favicon, the real title of the page and resolves the domain from any shortened links. # 22nd May 2009, 10:02 pm

TwitterAlikeExample—redis. Excellent example of how you design a moderately complex system against a scalable key-value store (in this case redis). Most “how to build Twitter” code examples fail to address the hard problem of scaling user inboxes, but this one tackles it head on. # 21st May 2009, 11:14 pm

peeping into memcached. “Peep uses ptrace to freeze a running memcached server, dump the internal key metadata, and return the server to a running state”—you can then load the resulting data in to MySQL using LOAD LOCAL INFILE and analyse it using standard SQL queries. # 20th April 2009, 6:35 pm

Sign in with Twitter. Intriguing: Twitter are now an OpenID-style identity provider... using OAuth. # 20th April 2009, 4:10 am

17-year-old claims responsibility for Twitter worm. It was a text book XSS attack—the URL on the user profile wasn’t properly escaped, allowing an attacker to insert a script element linking out to externally hosted JavaScript which then used Ajax to steal any logged-in user’s anti-CSRF token and use it to self-replicate in to their profile. # 12th April 2009, 7:22 pm

Twitter: blaming Ruby for their mistakes? The comments on the entry include replies from Twitter employees and the RabbitMQ consultant they brought in, and provide a full rebuttal to the various accusations of NIH that were thrown around recently. # 6th April 2009, 11:06 am

Mending The Bitter Absence of Reasoned Technical Discussion. Not at all surprised to see Alex Payne write this considering the low quality of discussion around anything technical to do with Twitter. # 5th April 2009, 7:59 pm

Streams, affordances, Facebook, and rounding errors. I asked Kellan about scaling activity streams the other day. Here he suggests the best technique is not to promise a perfect stream (like Twitter does)—Facebook used to get away with 80% loss of update messages, but their new redesign has changed the contract with their users. # 19th March 2009, 2:02 pm

Parallel merge sort in Erlang. Thoughts on an Erlang-y way of implementing a combined activity stream (e.g. Facebook and Twitter). Activity streams are a Really Hard Problem—as far as I know there’s no best practise for implementing them yet. # 15th March 2009, 1:36 pm

How search.twitter.com uses Varnish. Includes examples of the configuration options they use. # 2nd March 2009, 5:08 pm

Kestrel. Twitter’s Robey Pointer rewrote their Starling message queue in 1500 lines of Scala, adding reliable fetch (where consumers can confirm their receipt of an item) and blocking fetches, which reduce the need for consumers to poll for updates (and hence solve my only beef with the original Starling). I haven’t tried running this on a low spec VPS yet but it looks very promising. # 26th February 2009, 10:20 am

Oscars 2009: the interactive results | guardian.co.uk. My latest project for the Guardian, put together on very short notice. Updates live as the results are announced, and allows Twitter users to vote on their favourite for each category by sending a specially formatted message to @guardianfilm—jQuery and Ajax polling against S3 under the hood. # 23rd February 2009, 2:19 am

Twitter Don’t Click Exploit. Someone ran a successful ClickJacking exploit against Twitter users, using a transparent iframe holding the Twitter homepage with a status message fed in by a query string parameter. Thiss will definitely help raise awareness of ClickJacking! Twitter has now added framebusting JavaScript to prevent the exploit. # 12th February 2009, 7:56 pm