Simon Willison’s Weblog

Hookbox (via) For most web projects, I believe implementing any real-time comet features on a separate stack from the rest of the application makes sense—keep using Rails, Django or PHP for the bulk of the application logic, and offload any WebSocket or Comet requests to a separate stack built on top of something like Node.js, Twisted, EventMachine or Jetty. Hookbox is the best example of that philosophy I’ve yet seen—it’s a Comet server that makes WebHook requests back to your regular application stack to check if a user has permission to publish or subscribe to a given channel. “The key insight is that all application development with hookbox happens either in JavaScript or in the native language of the web application itself”. # 29th July 2010, 9:48 am

canto.js: An Improved HTML5 Canvas API (via) Improved is an understatement: canto adds jQuery-style method chaining, the ability to multiple coordinates to e.g. lineTo at once, relative coordinate methods (regular Canvas does everything in terms of absolute coordinates), the ability to use degrees instead of radians, a rounded corner shortcut, a more convenient .revert() method and a simple parser that can understand SVG path expressions! The only catch: it uses getters and setters so won’t work in IE. # 29th July 2010, 9:39 am

nodejitsu’s node-http-proxy (via) Exactly what I’ve been waiting for—a robust HTTP proxy library for Node that makes it trivial to proxy requests to a backend with custom proxy behaviour added in JavaScript. The example app adds an artificial delay to every request to simulate a slow connection, but other exciting potential use cases could include rate limiting, API key restriction, logging, load balancing, lint testing and more besides. # 28th July 2010, 11:34 pm

Multi-node: Concurrent NodeJS HTTP Server. Kris Zyp’s library for spawning multiple Node child processes (one per core is suggested) for concurrent request handling, taking advantage of Node’s child_process module. This alleviates the need to run multiple Node instances behind an nginx load balancer in order to take advantage of multiple cores. # 15th July 2010, 8:22 am

DNode: Asynchronous Remote Method Invocation for Node.js and the Browser. Mind-bendingly clever. DNode lets you expose a JavaScript function so that it can be called from another machine using a simple JSON-based network protocol. That’s relatively straight-forward... but DNode is designed for asynchronous environments, and so also lets you pass callback functions which will be translated in to references and used to make remote method invocations back to your original client. And to top it off, there’s a browser client library so you can perform the same trick over a WebSocket between a browser and a server. # 11th July 2010, 2:27 pm

Diffable: only download the deltas. JavaScript library for detecting and serving diffs to JavaScript rather than downloading large scripts every time a few lines of code are changed. “Using Diffable has reduced page load times in Google Maps by more than 1200 milliseconds (~25%). Note that this benefit only affects users that have an older version of the script in cache. For Google Maps that’s 20-25% of users.” # 11th July 2010, 12:19 pm

getlatlon.com commit dae961a... I’ve finally added an OpenStreetMap tab to getlatlon.com—here’s the diff, it turns out adding a custom OpenStreetMap layer to an existing Google Maps application only takes a few lines of boilerplate code. # 10th July 2010, 12:22 pm

Escaping regular expression characters in JavaScript (updated). The JavaScript regular expression meta-character escaping code I posted back in 2006 has some serious flaws—I’ve just posted an update to the original post. # 4th July 2010, 6:23 pm

jQuery.queueFn. “Execute any jQuery method or arbitrary function in the animation queue”. I’m surprised this isn’t baked in to jQuery itself—the plugin is only a few lines of code. # 30th June 2010, 12:59 pm

pdf.js. A JavaScript library for creating simple PDF files. Works (flakily) in your browser using a data:URI hack, but is also compatible with server-side JavaScript implementations such as Node.js. # 17th June 2010, 7:39 pm

Parsing file uploads at 500 mb/s with node.js. Handling file uploads is a real sweet spot for Node.js, especially now it has a high performance Buffer API for dealing with binary chunks of data. Felix Geisendörfer has released a new library called “formidable” which makes receiving file uploads (including HTML5 multiple uploads) easy, and uses some clever algorithmic tricks to dramatically speed up the processing of multipart data. # 2nd June 2010, 3:57 pm

tobeytailor’s gordon. Another Flash runtime in pure JavaScript project, released back in January. Not quite as advanced as Smokescreen yet (it doesn’t have an audio implementation) but already available as open source under an MIT license. # 29th May 2010, 11:57 am

Smokescreen demo: a Flash player in JavaScript. Chris Smoak’s Smokescreen, “a Flash player written in JavaScript”, is an incredible piece of work. It runs entirely in the browser, reads in SWF binaries, unzips them (in native JS), extracts images and embedded audio and turns them in to base64 encoded data:uris, then stitches the vector graphics back together as animated SVG. Open up the Chrome Web Inspector while the demo is running and you can see the SVG changing in real time. Smokescreen even implements its own ActionScript bytecode interpreter. It’s stated intention is to allow Flash banner ads to execute on the iPad and iPhone, but there are plenty of other interesting applications (such as news site infographics). The company behind it have announced plans to open source it in the near future. My one concern is performance—the library is 175 KB and over 8,000 lines of JavaScript which might cause problems on low powered mobile devices. # 29th May 2010, 11:32 am

Busting frame busting: a study of clickjacking vulnerabilities at popular sites (via) Fascinating and highly readable security paper from the Stanford Web Security Research group. Clickjacking can be mitigated using framebusting techniques, but it turns out that almost all of those techniques can be broken in various ways. Fun examples include double-nesting iframes so that the framebusting script overwrites the top-level frame rather than the whole window, and a devious attack against the IE and Chrome XSS filters which tricks them in to deleting the framebusting JavaScript by reflecting portions of it in the framed page’s URL. The authors suggest a new framebusting snippet that should be more effective, but sadly it relies on blanking out the whole page in CSS and making it visible again in JavaScript, making it inaccessible to browsers with JavaScript disabled. # 24th May 2010, 11:40 am

jed’s fab. Spectacular web framework for Node.js which, despite using nothing but regular JavaScript, has syntax that is easily confused with Lisp. General consensus at work is that truly understanding how this works is a crucial step on the path to JavaScript enlightenment. # 18th May 2010, 6:50 pm

Understanding node.js. A king providing orders to his army of servants is a much better analogy than my hyperactive squid. # 18th May 2010, 6:44 pm

The All-In-One Almost-Alphabetical No-Bullshit Guide to Detecting Everything. Appendix A of Dive Into HTML5. # 14th May 2010, 2:17 pm

Music Notation with HTML5 Canvas. A pretty decent effort at rendering musical notation using JavaScript and the canvas element. # 12th May 2010, 8:53 am

Realtime Election Tweets. Jay Caines-Gooby’s realtime election tweet service, using Node.js, nginx and WebSocket with a Flash fallback. # 6th May 2010, 9:20 pm

premasagar’s sandie. “Sandie is a simple method for loading external JavaScript files into a page without affecting the global scope, to avoid collisions between conflicting scripts”—works by loading the script in an invisible iframe (hence a new global scope) and then passing a reference to a callback function in the parent page. # 6th May 2010, 8:37 pm

Pure CSS3 Spiderman Cartoon w/ jQuery and HTML5. Great demo, though calling -webkit-animation HTML5 (or even CSS3) is a bit of a stretch... # 4th May 2010, 7:27 pm

A HTTP Proxy Server in 20 Lines of node.js. Proxying is definitely a sweet spot for Node.js. Peteris Krummins takes it a step further, adding host blacklists and an IP whitelist as configuration files and using Node’s watchFile method to automatically reload changes to them. # 28th April 2010, 1:24 pm

Lazy Load Plugin for jQuery. I’m using this jQuery plugin to save some bandwidth when people first view my Redis tutorial slides. It unobtrusively replaces images on a page with a placeholder graphic, then sets them to load automatically as the user scrolls down the page. # 26th April 2010, 12:02 am

tempalias.com development diary (via) tempalias.com is a e-mail forwarding service that lets you create an address that will only work for a few days (or a limited number of messages) and will forward messages on to your real account. It’s implemented using Node.js and Redis and the code is released under an MIT license. Philip Hofstetter, the developer, maintained a detailed development diary throughout which is worth reading if you’re interested in Node.js. # 23rd April 2010, 7:36 pm

jQuery special events. Ben Alman’s comprehensive guide to jQuery’s special events API, which allows you to register new kinds of events that can then be attached and detached using jQuery’s bind and unbind methods. Ben’s clickoutside event is a particularly useful example. # 17th April 2010, 9:08 pm

Step for Node.js. A further iteration on the attempts to make callback-based programming in Node.js easier to manage, this time making clever use of the ’this’ keyword to represent the next callback in the chain. # 13th April 2010, 11:02 am

What’s wrong with extending the DOM. Detailed explanation of the problems that crop up from extending built-in DOM objects using JavaScript, from Prototype developer kangax. Prototype 2.0 will be dropping this technique entirely—will MooTools follow suit? # 11th April 2010, 10:03 pm

jQuery UI: Trying to manipulate the position of a draggable mid-drag doesn’t seem to work. This has bitten me on two separate projects now—it’s the only problem I’ve had with jQuery UI’s draggables, which have otherwise been fantastic. # 11th April 2010, 9:59 pm

Preview: Freebase Gridworks (via) If my experience with government datasets has taught me anything, it’s that most datasets are collected by human beings (probably using Excel) and human beings are inconsistent. The first step in any data related project inevitably involves cleaning up the data. The Freebase team must run up against this all the time, and it looks like they’re tackling the problem head-on. Freebase Gridworks is just a screencast preview at the moment but an open source release is promised “within a month”—and the tool looks absolutely fantastic. DabbleDB-style data refactoring of spreadsheet data, running on your desktop but with the UI served in a browser. Full undo, a JavaScript-based expression language, powerful faceting and the ability to “reconcile” data against Freebase types (matching up country names, for example). I can’t wait to get my hands on this. # 27th March 2010, 6:43 pm

webhook-relay. Another of my experiments with Node.js: webhook-relay is a self-contained queue and webhook request sending agent. Your application can POST to it specifying a webhook alert to be sent off, and webhook-relay will place that request in an in-memory queue and send it on its own time, avoiding the need for your main application server to block until the outgoing request has been processed. # 19th March 2010, 10:17 am