Simon Willison’s Weblog

grammar.coffee (via) The annotated grammar for CoffeeScript, a new language that compiles to JavaScript developed by DocumentCloud’s Jeremy Ashkenas. The linked page is generated using Jeremy’s Docco tool for literate programming, also written in CoffeeScript. CoffeeScript itself is implemented in CoffeeScript, using a bootstrap compiler originally written in Ruby. 8th March 2010, 7:27 pm

twitter-text-conformance (via) This is a neat idea: Twitter have released open source libraries for parsing standard tweet syntax in Ruby and Java, but they’ve also released a set of YAML unit tests aimed at anyone who wants to implement the same parsing logic in other languages. 6th February 2010, 3:39 pm

rlisagor’s freshen. A Python clone of Ruby’s innovative Cucumber testing framework. Tests are defined as a set of plain-text scenarios, which are then executed by being matched against test functions decorated with regular expressions. Has anyone used this or Cucumber? I’m intrigued but unconvinced—are the plain text scenarios really a useful way of defining tests? 5th January 2010, 7:30 pm

clarity. A web interface for tailing and grepping the log files in /var/log, written in Ruby and EventMachine. 4th November 2009, 10:36 pm

Introducing Resque. A new background worker management queue developed at GitHub, using Redis for the persistence layer. The blog post explains both the design and the shortcomings of previous solutions at length. Within 24 hours of the release code an external developer, Adam Cooke, has completely reskinned the UI. 4th November 2009, 8:20 pm

How We Made GitHub Fast. Detailed overview of the new GitHub architecture. It’s a lot more complicated than I would have expected—lots of moving parts are involved in ensuring they can scale horizontally when they need to. Interesting components include nginx, Unicorn, Rails, DRBD, HAProxy, Redis, Erlang, memcached, SSH, git and a bunch of interesting new open source projects produced by the GitHub team such as BERT/Ernie and ProxyMachine. 21st October 2009, 9:14 pm

MySQL backups with EBS snapshots. Assaf Arkin’s 45 line ruby script shows how to lock tables / XFS freeze / create an EBS snapshot / unfreeze and unlock, with hourly snapshots preserved for the past 24 hours and daily snapshots for the past week. Is an EBS snapshot enough to restore your data to somewhere other than EC2 though? 13th October 2009, 12:34 pm

XSS Protection by Default in Rails 3.0. Fantastic news—congratulations, Rails core team. 8th October 2009, 4:35 pm

I like Unicorn because it’s Unix. Ryan Tomayko analyses Unicorn, a new, pre-forking Ruby HTTP server that makes extensive use of Unix syscalls and idioms, and asks why dynamic language programmers don’t take advantage of these more often. 7th October 2009, 11:42 am

cloud-crowd. New parallel processing worker/job queue system with a strikingly elegant architecture. The central server is an HTTP server that manages job requests, which are farmed out to a number of node HTTP servers which fork off worker processes to do the work. All communication is webhook-style JSON, and the servers are implemented in Sinatra and Thin using a tiny amount of code. The web-based monitoring interface is simply beautiful, using canvas to display graphs showing the system’s overall activity. 21st September 2009, 11:09 pm

homebrew. Exciting alternative to MacPorts for compiling software on OS X—homebrew avoids sudo and defines packages as simple Ruby scripts, shared and distributed using Git. 21st September 2009, 6:51 pm

Webhooks behind the firewall with Reverse HTTP. Hookout is a Ruby / rack adapter that lets you serve a web application from behind a firewall, by binding to a Reverse HTTP proxy running on the internet (such as the free one provided by reversehttp.net). Useful for far more than just webhooks, this means you can easily expose any Ruby web service to the outside world. An implementation of this as a general purpose proxy server would make it useful for applications written in any language. 22nd July 2009, 1:46 pm

Phusion Passenger for nginx. Passenger (aka mod_rails / mod_rack) enables easy deployment of Rails and Ruby apps under Apache... and the latest version adds support for nginx as well. It works as an HTTP proxy and process manager, spawning worker processes and forwarding HTTP requests to them via a request queue. It can also handle Python WSGI applications—anyone tried it out for that yet? 20th April 2009, 4:53 am

Twitter: blaming Ruby for their mistakes? The comments on the entry include replies from Twitter employees and the RabbitMQ consultant they brought in, and provide a full rebuttal to the various accusations of NIH that were thrown around recently. 6th April 2009, 11:06 am

Ruby on Rails 2.3 Release Notes. I’m impressed with how thoroughly Rails has embraced Rack (Ruby’s standardised web framework API, inspired by Python’s WSGI). 15th March 2009, 1:22 pm

CloudMade: A Summary of the Future of Mapping. CloudMade are now offering commercially supported APIs on top of OpenStreetMap, including geocoding, routing and tile access libraries in Python/Ruby/Java and a very neat theming tool that lets you design your own map styles. This is really going to kick innovation around OpenStreetMap up a notch. 17th February 2009, 11:25 am

juno. An ultra-lightweight Python web framework inspired by Ruby’s Sinatra. 4th February 2009, 10:48 am

Prawn (via) Really nice PDF generation library for Ruby, used to generate Dopplr’s beautiful end of year reports. 16th January 2009, 4:04 pm

why’s potion. why’s latest project is a small, fast language (JIT to x86/x86-64) which seems to take ideas from Ruby, Lua, Python and who knows where else. Everything is based around objects, closures and mixins, with the delightful inclusion of scoped mixins so you can modify an object only within a certain module (hence avoiding Ruby’s action-at-a-distance problems). 8th January 2009, 6:37 pm

Merb gets merged into Rails 3! Huge news. Of particular interest is the new focus on “framework agnosticism”, whereby Rails will aim to play well with people wishing to use alternative ORMs, template mechanisms and so forth. Rails has previously suffered from a reputation for getting in your way if you deviate from its opinions. 23rd December 2008, 8:32 pm

On packaging. James Bennett discusses the problems with setuptools (and ruby gems), and recommends Ian Bicking’s pip as a setuptools replacement. 14th December 2008, 4:57 pm

Reia. The most common complaint I see about Erlang is the syntax. Reia is a Python-style scripting language (with a dash of Ruby) that runs on the Erlang virtual machine. Looks promising. 25th September 2008, 6:12 pm

DoS vulnerability in REXML. Ruby’s REXML library is susceptible to the “billion laughs” denial of service attack where recursively nested entities expand a single entitity reference to a billion characters (kind of like the exploding zip file attack). Rails applications that process user-supplied XML should apply the monkey-patch ASAP; a proper gem update is forthcoming. 23rd August 2008, 11:11 am

Ruby’s Vulnerability Handling Debacle. The critical Ruby vulnerabilities are over a week old now but there’s still no good official patch (the security patches cause segfaults in Rails, leaving the community reliant on unofficial patches from third parties). Max Caceres has three takeaway lessons, the most important of which is to always keep a “last-known-good” branch to apply critical patches to. 2nd July 2008, 10:39 am

Whitespace Sensitivity. Amusingly, Ruby is actually far more sensitive about whitespace than Python is. 1st July 2008, 2:50 pm

Arbitrary code execution vulnerabilities in Ruby (via) Sounds nasty—integer overflows and unsafe allocs leading to memory corruption. Definite DoS and possible code injection as well. Upgrade to Ruby “1.8.5-p231, or 1.8.6-p230, or 1.8.7-p22” ASAP. 21st June 2008, 8:44 am

Spicing Up Embedded JavaScript. John Resig collects the various ways in which a JavaScript interpreter can be hosted by Python, PHP, Perl, Ruby and Java. There are full JS implementations in PHP, Perl and Java; Ruby and Python both have modules that use an embedded SpiderMonkey. 15th June 2008, 11:32 am

MagLev recap. Avi Bryant reports on the RailsConf demo of MagLev, a new closed-source Ruby implementation built with Gemstone (Smalltalk) technology that packs some impressive features (transaction based concurrency across multiple VMs) and exciting benchmarks (6-100x faster than the standard Ruby interpreter). 1st June 2008, 11:26 pm

Sneaking Ruby Through Google App Engine (and Other Strictly Python Places). In a characteristic stroke of genius, _why makes a solid initial attempt at compiling Ruby 1.9 source to Python 2.5 bytecode. 5th May 2008, 10:13 pm

Promise and Peril for Alternative Ruby Impls. Charles Nutter’s detailed and opinionated overview of the state of twelve different Ruby implementations (six of which are covered in detail). 27th April 2008, 2:18 pm

god—process and task monitoring done right. I have a long running animosity towards every process monitoring tool currently in existence; I’ll have to put this one through its paces and see if it sucks less. 29th March 2008, 10:20 pm

Monkeypatching is Destroying Ruby (via) Deliberately provocative title, but makes a well considered case for restrained use of monkey patching in Ruby. Cultural norms around monkey patching seem to me to be one of the core differences between the Ruby and Python communities. 22nd March 2008, 12:27 am

Windows Live ID Delegated Authentication. Would make life a lot simpler if they just supported OAuth, but at least they include sample code in Python, Ruby and PHP. 8th March 2008, 3:19 pm

RubyForge: Starling. “Starling is a light-weight persistent queue server that speaks the MemCache protocol. It was built to drive Twitter’s backend, and is in production across Twitter’s cluster.” 11th January 2008, 9:47 pm

Is your Rails app XSS safe? SafeErb is an interesting take on auto-escaping for Rails: it throws an exception if you try to render a string that hasn’t been untainted yet. 10th January 2008, 6:46 pm

Ruby 1.9—Right for You? Dave Thomas on the just-released Ruby 1.9. It’s a development release that breaks backwards compatibility in a few minor ways, but new features include the YARV virtual machine (hence significant speed improvements) and unicode support via associating encodings with bytestrings. 26th December 2007, 12:09 pm

stompserver. I think this is the lightweight message queue I’ve been looking for: written in Ruby and EventMachine, easy to set up (thanks to gems), interoperates perfectly with stomp.py. 14th December 2007, 4:40 pm

Two months with Ruby on Rails. Good rant—covers both the good and the bad. The first complaint is the lack of XSS protection by default in the template language. Django has the same problem, but the solution was 90% there when I saw Malcolm at OSCON. 9th October 2007, 12:23 pm

identity-matcher. Dopplr’s social network importing code (for Gmail, Twitter, Facebook and sites supporting Microformats), implemented as a Rails ActiveRecord plugin. 4th October 2007, 2:53 pm

Programming Nu (via) Interesting new programming language—Lisp style syntax, Ruby style semantics, built in Objective C bridge so you can access Cocoa APIs directly. 1st October 2007, 9:49 pm

The Rubinius Sprint. Sun are throwing a ton of resources at Ruby, because as Tim Bray says, “it’s not fast enough”. Imagine where they’d be if they’d invested this kind of support in Jython five years ago... 21st September 2007, 11:32 pm

Net::SSH revisited (via) Dependency injection (at least in Ruby) officially isn’t cool any more. 1st August 2007, 10:42 am

lwqueue. Lightweight cross-language message queue system, written in Perl with client libraries in Perl, Python and Ruby. 16th July 2007, 10:04 am

The CSS Redundancy Checker. A tool for checking your markup for outdated CSS rules that don’t match any of your HTML. We were discussing the need for something similar to this at Torchbox a few weeks ago. 6th July 2007, 12:02 pm

Mac OS X Leopard: UNIX. Leopard ships with DTrace, and it’s been hooked in to Java, Ruby, Python and Perl. 11th June 2007, 11:05 pm

The One True Object (Part 2). Jim Hugunin describes how the DLR let’s Python / JavaScript / Ruby talk to each other using a message passing abstraction. 5th May 2007, 1:27 am

Dynamic Language Runtime. Miguel de Icaza describes how Microsoft’s new Dynamic Language Runtime lets you call JavaScript and Visual Basic functions from Ruby. Looks like they beat Parrot to the punch. 3rd May 2007, 10:29 pm

Rack. “Rack provides an minimal interface between webservers supporting Ruby and Ruby frameworks”. Ruby’s equivalent of WSGI has just hit v0.1. 4th March 2007, 8:49 pm

method_missing: best saved for last. My least favourite thing about Ruby is the cultural tendency towards introducing weird new bugs in other people’s code. 7th February 2007, 2:16 pm

Apache Solr 1.1. Solr is the search Web Service built on top of Lucene. The latest release introduces JSON, Python and Ruby response formats in addition to XML. 13th January 2007, 1:16 am

Stemtags is back, thanks to Camping. Nice example of a throw-away single script web app, 10th December 2006, 8:51 pm