Simon Willison’s Weblog

PaintbrushJS. Impressive open source JavaScript library from Dave Shea for applying image filters (sharpen, blur, emboss, greyscale etc) to the canvas element. # 9th October 2010, 11:53 am

What is a Polyfill? Useful new term: a Polyfill is “a shim that mimics a future API providing fallback functionality to older browsers”. # 9th October 2010, 11:48 am

Schneier on Stuxnet. Stuxnet now rivals Wikileaks as the real life plot most likely to have leaked from science fiction. # 9th October 2010, 10:57 am

What is the Open Web? Tantek Çelik describes the three pillars of the open web: open publishing of content, freedom to code and implement the standards needed to access that content and open access to that content over an unfiltered internet. # 9th October 2010, 10:47 am

“Last I heard Google’s search index was sharded by document rather than by term.”. Fascinating comment by jasonwatkinspdx on Hacker News. # 8th October 2010, 8:03 pm

The 100-year leap. John Graham-Cumming recounts the history of Charles Babbage’s Difference Engine and Analytical Engine, and proposes a project to build a working Analytical Engine 170 years after its invention (the machine built by the Science Museum in London is the Difference Engine). # 6th October 2010, 9:26 am

My First Week with the iPhone. A blind user describes the experience of using VoiceOver on the iPhone, including the joy of discovering the Color Identifier app which speaks the names of colours picked up by the iPhone’s camera. “ I used color cues to find my pumpkin plants, by looking for the green among the brown and stone. I spent ten minutes looking at my pumpkin plants, with their leaves of green and lemon-ginger.” # 3rd October 2010, 12:20 pm

Facebook’s Instant Personalization: An Analysis of Fundamental Privacy Flaws (via) Oh FFS. “Instant Personalization” means you visit one of Facebook’s “partner websites” and Facebook instantly tells them your full identity and gives them access to full Facebook connect functionality—without you performing any action other than visiting the site. This will not end well. # 2nd October 2010, 11:53 pm

Velocity: Forcing Gzip Compression. Almost every browser supports gzip these days, but 15% of web requests have had their Accept-Encoding header stripped or mangled, generally due to poorly implemented proxies or anti-virus software. Steve Souders passes on a trick used by Google Search, where an iframe is used to test the browser’s gzip support and set a cookie to force gzipping of future pages. # 30th September 2010, 5:45 pm

evercookie—virtually irrevocable persistent cookies (via) Mischievous genius from the chap who created the MySpace worm—evercookie attempts to set an irrevocable cookie using a whole bunch of different methods, including “storing cookies in RGB values of auto-generated, force-cached PNGs using HTML5 Canvas tag to read pixels back out” and an extremely clever scheme built on top of the web history CSS visited link colour vulnerability. # 23rd September 2010, 1:35 am

Google Chrome Frame: Stable and Speedy (via) “Today, we’re very happy to take the Beta tag off of Google Chrome Frame and promote it to the Stable channel.”—MSI installer included, for IT administrators to easily deploy Chrome Frame to multiple machines. # 23rd September 2010, 1:34 am

I pushed 20 more of my projects to GitHub. Some great Node.js stuff here from Peteris Krumins, including modules for processing PNG, JPEG and animated GIFs. # 23rd September 2010, 1:18 am

ijson. A SAX-style streaming JSON parser for Python, using ctypes to talk to the yajl C library. # 22nd September 2010, 9:59 pm

A Gentle Introduction to CouchDB for Relational Practitioners. By “High Performance MySQL” author Baron Schwartz—a smart, concise overview that touches pretty much everything that’s interesting about CouchDB. # 22nd September 2010, 9:51 pm

Creating Shazam in Java. Using a Fast Fourier Transformation. # 22nd September 2010, 9:39 pm

Welcome to Lanyrd | The Lanyrd Blog. We’ve started a blog for Lanyrd, our social conference directory project. We’re off to a great start: “Lanyrd is now listing 1,508 conferences and 5,167 individual speaker profiles. 5,637 people have signed in to the site and made 13,293 edits to our data.” # 11th September 2010, 9:32 pm

ZeroMQ: Modern and Fast Networking Stack. I get ZeroMQ now. I was having trouble figuring out how it differed from things like RabbitMQ—it turns out it’s an entirely new low-level socket abstraction, designed to make common socket programming tasks like message sending/receiving and publish/subscribe a whole lot easier than dealing with raw BSD sockets. # 5th September 2010, 7:41 pm

Vox is closing on September 30, 2010. One month seems like very short notice for closing a service of this size, especially since it functions as an OpenID provider so in addition to migrating their content away users may need to sign in to other services and set up an alternative form of authentication. UPDATE: From the comments, Vox accounts that migrate to TypePad will also have their OpenID migrated, and TypePad will continue to serve OpenID requests for old vox.com addresses. Smart solution. # 3rd September 2010, 8:50 am

The Seven Secrets of Successful Data Scientists. Some sensible advice, including pick the right sized tool, compress everything, split up your data, use open source and run the analysis where the data is. # 3rd September 2010, 12:36 am

Setting up Munin on Ubuntu. Useful guide to setting up my favourite graphing/monitoring tool for personal projects. # 1st September 2010, 2:05 pm

RasterWeb: Lanyrd. Pete Prodoehl calls me out on Lanyrd’s integration with the Twitter auth API at the expense of OpenID. I’ve posted a comment with my justification—essentially, tying to Twitter’s ecosystem means I can actually implement the features I’ve been talking about building on top of OpenID for years, with far less engineering effort. # 31st August 2010, 8:49 pm

Lanyrd—the social conference directory. Nat and my new project, launched today and doing pretty well despite some early server hiccups. Sign in with Twitter to see conferences that your friends are speaking at, attending or tracking, then add your own events. We’re particularly keen on helping people build up a detailed profile of their previous talks, so adding older conferences is encouraged. # 31st August 2010, 7:41 pm

LWPx::ParanoidAgent. Every programming language needs an equivalent of this library—a robust, secure way to make HTTP requests against URLs from untrusted sources without risk of tarpits, internal network access, socket starvation, weird server errors, or other nastiness. # 31st August 2010, 2:30 am

What is the history of Django? I’ve been playing with Quora—it’s a really neat twist on the question-and-answer format, which makes great use of friends, followers and topics and has some very neat live update stuff going on (using Comet on top of Tornado). I just posted quite a long answer to a question about the history of Django. # 24th August 2010, 5:31 pm

Readme Driven Development (via) Tom Preston-Werner advocates for writing the readme before any other code. “Until you’ve written about your software, you have no idea what you’ll be coding.” # 23rd August 2010, 8:20 pm

Using Freebase Gridworks to Create Linked Data. A very handy tutorial from data.gov.uk’s Jeni Tennison. # 23rd August 2010, 8:11 pm

PNGStore—Embedding compressed CSS & JavaScript in PNGs. Cal did some further analysis on the CSS/JS to PNG compression trick (including producing some interesting images of jQuery compressed using different image packing techniques) and found it to be slightly less effective than regular GZipping. # 23rd August 2010, 9:47 am

10K Apart Contest: Cheating by Compressing Your JavaScript and CSS to PNG Images. Fascinating hack: transform your JS and CSS in to coloured pixels, save the result as a PNG to benefit from PNG’s built in compression algorithms, then read the data back out of the PNG and convert it back to text using JavaScript and canvas—all to reduce the on-disk filesize when entering the 10K app competition. Alex’s GithubFinder entry is worth checking out too. # 23rd August 2010, 9:45 am

Undelete! How to undelete a file accidentally removed using rm on Linux, by grepping through the raw bytes on the hard drive searching for a unique string that was contained in the file. “grep -a -B 25 -A 100 ’some string in the file’ /dev/sda1 > results.txt” # 21st August 2010, 10:56 am

Polymaps. Absurdly classy: “a JavaScript library for image- and vector-tiled maps using SVG”. It can pull in image tiles from sources such as OpenStreetMap, then overlay SVG paths specified using GeoJSON. The demos make use of GeoJSON tiles for US states and counties hosted on AppEngine. The library is developed by Stamen and SimpleGeo, and released under a BSD license. SVG support in the browser is required. # 20th August 2010, 6:46 pm