Simon Willison’s Weblog

Porting Flickr to YUI 3: Lessons in Performance (at YUIConf 2010). Some very interesting tips here. The new Flickr photo pages suffered from what I’ve been calling “Flash of Un-Behavioured Content”, where slow loading JavaScript results in poor behaviour from some UI controls. They started using “Action Queueing”, where a small JS stub ensures a loading indicator is shown for clicks on features that have not yet fully loaded. Also, it turns out some corporate firewalls (Sonicwall in particular) dislike URLs over 1600 characters, and filter out any URL with xxx in it. # 10th November 2010, 6:33 pm

jQuery 1.4.3 Released. Once again, the thing that impresses me most about this jQuery release is how stable the core API is. Hardly any new methods added, but the existing methods are made faster, more flexible and more predictable. The same as been true for the past several releases as well. It just keeps getting more and more polished. # 17th October 2010, 12:15 am

Annotated backbone.js. Literate programming. # 13th October 2010, 5:24 pm

Backbone.js. As should be expected for a DocumentCloud project, Backbone is a concise, elegant and educational take on the JavaScript MVC pattern. Depends on Underscore.js and plays well with jQuery. # 13th October 2010, 5:23 pm

PaintbrushJS. Impressive open source JavaScript library from Dave Shea for applying image filters (sharpen, blur, emboss, greyscale etc) to the canvas element. # 9th October 2010, 11:53 am

10K Apart Contest: Cheating by Compressing Your JavaScript and CSS to PNG Images. Fascinating hack: transform your JS and CSS in to coloured pixels, save the result as a PNG to benefit from PNG’s built in compression algorithms, then read the data back out of the PNG and convert it back to text using JavaScript and canvas—all to reduce the on-disk filesize when entering the 10K app competition. Alex’s GithubFinder entry is worth checking out too. # 23rd August 2010, 9:45 am

Polymaps. Absurdly classy: “a JavaScript library for image- and vector-tiled maps using SVG”. It can pull in image tiles from sources such as OpenStreetMap, then overlay SVG paths specified using GeoJSON. The demos make use of GeoJSON tiles for US states and counties hosted on AppEngine. The library is developed by Stamen and SimpleGeo, and released under a BSD license. SVG support in the browser is required. # 20th August 2010, 6:46 pm

Closure Compiler Service (via) A hosted version of the Google Closure Compiler (JavaScript minifier) running on App Engine. It has both a user interface and a REST API, which means you can use it as part of an automated build process without needing to set up a local copy of the software. # 9th August 2010, 1:17 pm

Hookbox (via) For most web projects, I believe implementing any real-time comet features on a separate stack from the rest of the application makes sense—keep using Rails, Django or PHP for the bulk of the application logic, and offload any WebSocket or Comet requests to a separate stack built on top of something like Node.js, Twisted, EventMachine or Jetty. Hookbox is the best example of that philosophy I’ve yet seen—it’s a Comet server that makes WebHook requests back to your regular application stack to check if a user has permission to publish or subscribe to a given channel. “The key insight is that all application development with hookbox happens either in JavaScript or in the native language of the web application itself”. # 29th July 2010, 9:48 am

canto.js: An Improved HTML5 Canvas API (via) Improved is an understatement: canto adds jQuery-style method chaining, the ability to multiple coordinates to e.g. lineTo at once, relative coordinate methods (regular Canvas does everything in terms of absolute coordinates), the ability to use degrees instead of radians, a rounded corner shortcut, a more convenient .revert() method and a simple parser that can understand SVG path expressions! The only catch: it uses getters and setters so won’t work in IE. # 29th July 2010, 9:39 am

nodejitsu’s node-http-proxy (via) Exactly what I’ve been waiting for—a robust HTTP proxy library for Node that makes it trivial to proxy requests to a backend with custom proxy behaviour added in JavaScript. The example app adds an artificial delay to every request to simulate a slow connection, but other exciting potential use cases could include rate limiting, API key restriction, logging, load balancing, lint testing and more besides. # 28th July 2010, 11:34 pm

Multi-node: Concurrent NodeJS HTTP Server. Kris Zyp’s library for spawning multiple Node child processes (one per core is suggested) for concurrent request handling, taking advantage of Node’s child_process module. This alleviates the need to run multiple Node instances behind an nginx load balancer in order to take advantage of multiple cores. # 15th July 2010, 8:22 am

DNode: Asynchronous Remote Method Invocation for Node.js and the Browser. Mind-bendingly clever. DNode lets you expose a JavaScript function so that it can be called from another machine using a simple JSON-based network protocol. That’s relatively straight-forward... but DNode is designed for asynchronous environments, and so also lets you pass callback functions which will be translated in to references and used to make remote method invocations back to your original client. And to top it off, there’s a browser client library so you can perform the same trick over a WebSocket between a browser and a server. # 11th July 2010, 2:27 pm

Diffable: only download the deltas. JavaScript library for detecting and serving diffs to JavaScript rather than downloading large scripts every time a few lines of code are changed. “Using Diffable has reduced page load times in Google Maps by more than 1200 milliseconds (~25%). Note that this benefit only affects users that have an older version of the script in cache. For Google Maps that’s 20-25% of users.” # 11th July 2010, 12:19 pm

getlatlon.com commit dae961a... I’ve finally added an OpenStreetMap tab to getlatlon.com—here’s the diff, it turns out adding a custom OpenStreetMap layer to an existing Google Maps application only takes a few lines of boilerplate code. # 10th July 2010, 12:22 pm

Escaping regular expression characters in JavaScript (updated). The JavaScript regular expression meta-character escaping code I posted back in 2006 has some serious flaws—I’ve just posted an update to the original post. # 4th July 2010, 6:23 pm

jQuery.queueFn. “Execute any jQuery method or arbitrary function in the animation queue”. I’m surprised this isn’t baked in to jQuery itself—the plugin is only a few lines of code. # 30th June 2010, 12:59 pm

pdf.js. A JavaScript library for creating simple PDF files. Works (flakily) in your browser using a data:URI hack, but is also compatible with server-side JavaScript implementations such as Node.js. # 17th June 2010, 7:39 pm

Parsing file uploads at 500 mb/s with node.js. Handling file uploads is a real sweet spot for Node.js, especially now it has a high performance Buffer API for dealing with binary chunks of data. Felix Geisendörfer has released a new library called “formidable” which makes receiving file uploads (including HTML5 multiple uploads) easy, and uses some clever algorithmic tricks to dramatically speed up the processing of multipart data. # 2nd June 2010, 3:57 pm

tobeytailor’s gordon. Another Flash runtime in pure JavaScript project, released back in January. Not quite as advanced as Smokescreen yet (it doesn’t have an audio implementation) but already available as open source under an MIT license. # 29th May 2010, 11:57 am

Smokescreen demo: a Flash player in JavaScript. Chris Smoak’s Smokescreen, “a Flash player written in JavaScript”, is an incredible piece of work. It runs entirely in the browser, reads in SWF binaries, unzips them (in native JS), extracts images and embedded audio and turns them in to base64 encoded data:uris, then stitches the vector graphics back together as animated SVG. Open up the Chrome Web Inspector while the demo is running and you can see the SVG changing in real time. Smokescreen even implements its own ActionScript bytecode interpreter. It’s stated intention is to allow Flash banner ads to execute on the iPad and iPhone, but there are plenty of other interesting applications (such as news site infographics). The company behind it have announced plans to open source it in the near future. My one concern is performance—the library is 175 KB and over 8,000 lines of JavaScript which might cause problems on low powered mobile devices. # 29th May 2010, 11:32 am

Busting frame busting: a study of clickjacking vulnerabilities at popular sites (via) Fascinating and highly readable security paper from the Stanford Web Security Research group. Clickjacking can be mitigated using framebusting techniques, but it turns out that almost all of those techniques can be broken in various ways. Fun examples include double-nesting iframes so that the framebusting script overwrites the top-level frame rather than the whole window, and a devious attack against the IE and Chrome XSS filters which tricks them in to deleting the framebusting JavaScript by reflecting portions of it in the framed page’s URL. The authors suggest a new framebusting snippet that should be more effective, but sadly it relies on blanking out the whole page in CSS and making it visible again in JavaScript, making it inaccessible to browsers with JavaScript disabled. # 24th May 2010, 11:40 am

jed’s fab. Spectacular web framework for Node.js which, despite using nothing but regular JavaScript, has syntax that is easily confused with Lisp. General consensus at work is that truly understanding how this works is a crucial step on the path to JavaScript enlightenment. # 18th May 2010, 6:50 pm

Understanding node.js. A king providing orders to his army of servants is a much better analogy than my hyperactive squid. # 18th May 2010, 6:44 pm

The All-In-One Almost-Alphabetical No-Bullshit Guide to Detecting Everything. Appendix A of Dive Into HTML5. # 14th May 2010, 2:17 pm

Music Notation with HTML5 Canvas. A pretty decent effort at rendering musical notation using JavaScript and the canvas element. # 12th May 2010, 8:53 am

Realtime Election Tweets. Jay Caines-Gooby’s realtime election tweet service, using Node.js, nginx and WebSocket with a Flash fallback. # 6th May 2010, 9:20 pm

premasagar’s sandie. “Sandie is a simple method for loading external JavaScript files into a page without affecting the global scope, to avoid collisions between conflicting scripts”—works by loading the script in an invisible iframe (hence a new global scope) and then passing a reference to a callback function in the parent page. # 6th May 2010, 8:37 pm

Pure CSS3 Spiderman Cartoon w/ jQuery and HTML5. Great demo, though calling -webkit-animation HTML5 (or even CSS3) is a bit of a stretch... # 4th May 2010, 7:27 pm

A HTTP Proxy Server in 20 Lines of node.js. Proxying is definitely a sweet spot for Node.js. Peteris Krummins takes it a step further, adding host blacklists and an IP whitelist as configuration files and using Node’s watchFile method to automatically reload changes to them. # 28th April 2010, 1:24 pm