Simon Willison’s Weblog

I pushed 20 more of my projects to GitHub. Some great Node.js stuff here from Peteris Krumins, including modules for processing PNG, JPEG and animated GIFs. # 23rd September 2010, 1:18 am

nodejitsu’s node-http-proxy (via) Exactly what I’ve been waiting for—a robust HTTP proxy library for Node that makes it trivial to proxy requests to a backend with custom proxy behaviour added in JavaScript. The example app adds an artificial delay to every request to simulate a slow connection, but other exciting potential use cases could include rate limiting, API key restriction, logging, load balancing, lint testing and more besides. # 28th July 2010, 11:34 pm

Multi-node: Concurrent NodeJS HTTP Server. Kris Zyp’s library for spawning multiple Node child processes (one per core is suggested) for concurrent request handling, taking advantage of Node’s child_process module. This alleviates the need to run multiple Node instances behind an nginx load balancer in order to take advantage of multiple cores. # 15th July 2010, 8:22 am

DNode: Asynchronous Remote Method Invocation for Node.js and the Browser. Mind-bendingly clever. DNode lets you expose a JavaScript function so that it can be called from another machine using a simple JSON-based network protocol. That’s relatively straight-forward... but DNode is designed for asynchronous environments, and so also lets you pass callback functions which will be translated in to references and used to make remote method invocations back to your original client. And to top it off, there’s a browser client library so you can perform the same trick over a WebSocket between a browser and a server. # 11th July 2010, 2:27 pm

pdf.js. A JavaScript library for creating simple PDF files. Works (flakily) in your browser using a data:URI hack, but is also compatible with server-side JavaScript implementations such as Node.js. # 17th June 2010, 7:39 pm

Parsing file uploads at 500 mb/s with node.js. Handling file uploads is a real sweet spot for Node.js, especially now it has a high performance Buffer API for dealing with binary chunks of data. Felix Geisendörfer has released a new library called “formidable” which makes receiving file uploads (including HTML5 multiple uploads) easy, and uses some clever algorithmic tricks to dramatically speed up the processing of multipart data. # 2nd June 2010, 3:57 pm

jed’s fab. Spectacular web framework for Node.js which, despite using nothing but regular JavaScript, has syntax that is easily confused with Lisp. General consensus at work is that truly understanding how this works is a crucial step on the path to JavaScript enlightenment. # 18th May 2010, 6:50 pm

Understanding node.js. A king providing orders to his army of servants is a much better analogy than my hyperactive squid. # 18th May 2010, 6:44 pm

Realtime Election Tweets. Jay Caines-Gooby’s realtime election tweet service, using Node.js, nginx and WebSocket with a Flash fallback. # 6th May 2010, 9:20 pm

A HTTP Proxy Server in 20 Lines of node.js. Proxying is definitely a sweet spot for Node.js. Peteris Krummins takes it a step further, adding host blacklists and an IP whitelist as configuration files and using Node’s watchFile method to automatically reload changes to them. # 28th April 2010, 1:24 pm

tempalias.com development diary (via) tempalias.com is a e-mail forwarding service that lets you create an address that will only work for a few days (or a limited number of messages) and will forward messages on to your real account. It’s implemented using Node.js and Redis and the code is released under an MIT license. Philip Hofstetter, the developer, maintained a detailed development diary throughout which is worth reading if you’re interested in Node.js. # 23rd April 2010, 7:36 pm

webhook-relay. Another of my experiments with Node.js: webhook-relay is a self-contained queue and webhook request sending agent. Your application can POST to it specifying a webhook alert to be sent off, and webhook-relay will place that request in an in-memory queue and send it on its own time, avoiding the need for your main application server to block until the outgoing request has been processed. # 19th March 2010, 10:17 am

Node.js, redis, and resque (via) Paul Gross has been experimenting with Node.js proxies for allowing web applications to be upgraded without missing any requests. Here he places all incoming HTTP requests in a redis queue, then has his backend Rails servers consume requests from the queue and push the responses back on to a queue for Node to deliver. When the backend application is upgraded, requests remain in the queue and users see a few seconds of delay before their request is handled. It’s not production ready yet (POST requests aren’t handled, for example) but it’s a very interesting approach. # 28th February 2010, 11:02 pm

kriszyp’s node-promise. Another elegant approach to managing asynchronous flows in Node, including running things both in parallel and serial. # 28th February 2010, 3:50 pm

“Do” it fast! Tim Caswell’s Do library has been upgraded for compatibility with Node v0.1.30, and now has a clever Do.convert() method which wraps Node’s low-level APIs with the Do libraries “continuable” abstraction. # 22nd February 2010, 7:02 pm

node-v0.1.30 (via) A very significant new release of Node.js: the Twisted/Dojo-style Promise abstraction has been removed entirely, causing backwards incompatible changes to a bunch of core APIs. This means the pseudo-blocking Promise.wait() method is gone too, making it even harder to accidentally block your event loop. Instead, user-level libraries are encouraged to add Promise-style abstractions. I’m pleased to see Node sticking to the low-level stuff. # 22nd February 2010, 7 pm

do. A library for Node that adds a higher level abstraction for dealing with chained and parallel callbacks. # 17th February 2010, 5:43 pm

How To Node. New blog about Node.js, with a superb series of tutorials aimed at both experienced and new JavaScript developers. The stuff on managing callbacks (including running them in both series and parallel) is pretty eye-opening. # 17th February 2010, 5:42 pm

dogproxy. Another of my experiments with Node.js—this is a very simple HTTP proxy which addresses the dog pile effect (also known as the thundering herd) by watching out for multiple requests for a URL that is currently “in flight” and bundling them together. # 3rd February 2010, 1:05 pm

Plurk: Instant conversations using Comet (via) Plurk’s comet implementation sounds pretty amazing. They’re using a single quad-core server with 32GB of RAM running 8 Node.js instances to serve long-polled comet to 100,000+ simultaneous users. They switched to Node from Java JBoss/Netty and found the new solution used 10 times less memory. # 1st February 2010, 10:13 am

Hot Code Loading in Node.js. Blaine Cook’s patch for Node.js that enables Erlang-style hot code loading, so you can switch out your application logic without restarting the server or affecting existing requests. This could make deploying new versions of Node applications trivial. I’d love to see a Node hosting service that allows you to simply upload a script file and have it execute on the Web. # 31st January 2010, 1:57 pm