Simon Willison’s Weblog

Subscribe

July 2009

July 9, 2009

App Engine outage postmortem. Interesting peek behind the scenes. The primary cause of the error was a bug in a GFS (Google File System) Master server caused by a MapReduce process sending a malformed filehandle, reminiscent of the error which took down S3 last year.

# 12:49 pm / s3, google, appengine, downtime, gfs

Unlike progressive downloads, HTTP Live Streaming actually does stream content in real time, although there can be a latency of as much as 30 seconds. [...] the content to be broadcast is encoded into an MPEG transport stream and chopped into segments that are around ten seconds long. Rather than getting a continuous stream of new data over RTSP, the new protocol simply asks for the first couple clips, then asks for additional clips as needed. This works great through firewalls, and doesn't require any special servers because any standard web server can deliver the chopped up video segments.

Prince McLean

# 12:52 pm / apple, httplivestreaming, video, streaming, realtimeweb, mpeg

Social Media Icons. Paul Robert Lloyd: “ In the past I’ve used site favicons, but these can often be visually inconsistent”—so he’s put together a tasty set of icons for different social websites with a consistent visual feel, available in four different sizes.

# 4:38 pm / paul-robert-lloyd, design, icons, social-media

July 10, 2009

Google Will Eat Itself. “We generate money by serving Google text advertisments on a network of hidden Websites. With this money we automatically buy Google shares. We buy Google via their own advertisment!”

# 12:15 pm / google, clickfraud

July 11, 2009

HTML 5 Parsing. Firefox nightlies include a new parser that implements the HTML5 parsing algorithm (disabled by default), which uses C++ code automatically generated from Henri Sivonen’s Java parser first used in the HTML5 validator.

# 11:36 pm / john-resig, browsers, firefox, henrisivonen, html5, mozilla, parsing, validator

July 13, 2009

Twenty questions about the GPL. Jacob kicks off a fascinating discussion about GPLv3.

# 11:59 pm / gpl, gpl3, jacob-kaplan-moss, open-source, licenses

July 14, 2009

Meta Is Murder. I hadn’t realised how important MetaTalk was in ensuring high quality discussions on MetaFilter, by ensuring that meta-discussions happened somewhere else. Speaking of which, happy birthday MetaFilter.

# 7:34 pm / metafilter, metatalk, metadiscussions, blogging, jeff-atwood

July 15, 2009

Slouching towards Bethlehem. Photos of the various installations that contributed to the construction of the first atom bomb.

# 10:19 am / nuclear, atombomb, history, photos

You should follow me on Twitter. Dustin Curtis did a simple A/B testing experiment on his blog and found that the text “you should follow me on Twitter” had the highest click-through rate—173% more effective than “I’m on Twitter”.

# 10:43 am / twitter, copy, dustincurtis, ab-testing, buckettesting

July 16, 2009

Google’s Chiller-less Data Center. Google are operating an outside data center in Belgium with no chillers (refrigeration units used to cool water, but at a high cost in energy) making “local weather forecasting a larger factor in its data center management”. On the 10 or so days of the year when Belgium is too warm, they can simply shut down the data center and shift the workload elsewhere.

# 9:50 am / google, environment, energy, chillers, cooling, datacenters

I propose that the World Wide Web would serve well as a framework for structuring much of the academic Computer Science curriculum. A study of the theory and practice of the Web’s technologies would traverse many key areas of our discipline.

Tim Bray

# 10:16 am / web, tim-bray, computer-science, education

Keyspace. Yet Another Key-Value Store—this one focuses on high availability, with one server in the cluster serving as master (and handling all writes), and the paxos algorithm handling replication and ensuring a new master can be elected should the existing master become unavailable. Clients can chose to make dirty reads against replicated servers or clean reads by talking directly to the master. Underlying storage is BerkeleyDB, and the authors claim 100,000 writes/second. Released under the AGPL.

# 10:30 am / keyvaluepairs, keyspace, databases, agpl, berkeleydb, scaling, replication, paxos

IanVisits: London Events Calendar. Ian Mansfield maintains a superb calendar of cultural (and geeky) events in London. Lectures, tours, bat walks, film screenings... did you know there’s a Festival of Model Tramways this weekend?

# 7:22 pm / ian-mansfield, ian-visits, calendar, events, london

Curating conversations. Chris Thorpe has open-sourced the Guardian’s moderated Twitter backchannel app, for displaying back channels at high profile (and hence high potential for abuse) events. It’s a Python application that runs on App Engine.

# 7:34 pm / guardian, twitter, chris-thorpe, python, appengine

Nmap 5.00 Release Notes. Released today, “the most important Nmap release since 1997”. New features include Ncat, a powerful netcat alternative, Ndiff, a utility for comparing scan results so you can spot changes to your network, and a new Nmap Scripting Engine using Lua.

# 7:40 pm / nmap, ndiff, ncat, releases, netcat, security, tools, lua

Teaching users to be secure is a shared responsibility

Ryan Janssen: Why an OAuth iframe is a Great Idea.

[... 570 words]

NaCl: Networking and Cryptography library. A new high level cryptography library. “NaCl advances the state of the art by improving security, by improving usability and by improving speed.” Ambitious claims, but DJB is one of the core maintainers.

# 8:24 pm / nacl, cryptography, djb, security

Why an OAuth iframe is a Great Idea. Because users should a) learn to be phished and b) not even be given the option to avoid being phished if they know what they’re doing? No, no and thrice no. If you want to improve the experience, use a popup window so the user can still see the site they are signing in to in the background.

# 8:29 pm / oauth, security, phishing, iframes

July 17, 2009

TurboGears on Sourceforge. Sourceforge recently relaunched, powered by TurboGears 2 and MongoDB. Mark Ramm has the details.

# 2:30 am / sourceforge, mark-ramm, turbogears, turbogears2, mongodb

Announcing Alice and Wonderland. Continuing the RabbitMQ “stuff to do with rabbits” naming convention, Alice is a RESTful interface to RabbitMQ which exposes information about vhosts/queues/users/exchanges/etc as JSON. Wonderland is a web UI for RabbitMQ implemented as a pure Ajax application which calls Alice.

# 9:12 am / aliceinwonderland, rabbitmq, alice, wonderland, rest, json, ajax, javascript, message-queues, queues

Popfly Shutting Down. Yet another reminder that building stuff on a closed-source platform (especially a hosted service) is risky business, even from a vendor as large as Microsoft. This certainly won’t help them make the case for Azure.

# 9:32 am / open-source, closedsource, microsoft, azure, popfly, sharecropping

Where was the ’editorial viewpoint’ at the News Innovation unconference? Martin Belam points out that a problem with unconferences when applied to audiences outside the technology world is that techies who know how the system operates will inadvertently take over the event, skewing the conversation towards technical topics. Not an insurmountable problem, but one that organisers should probably take in to account.

# 10:52 am / conferences, martinbelam, unconferences, events, newsinnovation

Farewell to Mashup Editor. It’s not just Microsoft Popfly that’s shutting down—Google Mashup Editor will be gone in four weeks time (this was announced in January). You get to keep your code, but I don’t know enough about Mashup Editor to know if the code is usable once the system has shut down.

# 1:05 pm / googlemashupeditor, google, microsoft, popfly, sharecropping

Her Majesty The Queen will see the Swan Upping ceremony between Bovney Lock and Oakley Court on the River Thames, on the 20th July 2009. This is the first occasion that The Queen has witnessed the annual event.

Govt. Press Service

# 10:13 pm / thequeen, swanupping, ian-mansfield

Memcached 1.4.0 released. The big new feature is the (optional) binary protocol, which enables other features such as CAS-everywhere and efficient client-side replication. Maintainer Dustin Sallings has also released some useful sounding EC2 instances which automatically assign nearly all of their RAM to memcached on launch and shouldn’t need any further configuration.

# 10:26 pm / memcached, dustinsallings, binary, cas, ec2, ami, caching, performance, scaling

July 20, 2009

The Anatomy Of The Twitter Attack. Long-winded explanation of the recent Twitter break-in, but you can scroll to the bottom for a numbered list summary. The attacker first broke in to a Twitter employee’s personal Gmail account by “recovering” it against an expired Hotmail account (which the attacker could hence register themselves). They gained access to more passwords by searching for e-mails from badly implemented sites that send you your password in the clear.

# 12:55 am / gmail, hotmail, passwords, security, twitter

Most journalists have grown up with a fortress mindset. They have lived and worked in proud institutions with thick walls. Their daily knightly task has been simple: to battle journalists from other fortresses. But the fortresses are crumbling and courtly jousts with fellow journalists are no longer impressing the crowds.

Peter Horrocks

# 5:20 pm / bbc, journalism, peter-horrocks, newspapers

July 21, 2009

Early Day Motion to support Bletchley Park Museum. Time to fire up WriteToThem.com and drop your MP a friendly note of encouragement.

# 1:56 pm / politics, bletchleypark, edparsons

Reverse HTTP Demo (via) This is a bit of a brain teaser—a web server running in JavaScript in your browser which uses long polling comet to respond to incoming HTTP requests channelled through a “Reverse HTTP” proxy.

# 3:54 pm / reversehttp, comet, javascript

moddims (via) Apache 2 module which exposes ImageMagick as a URL-driven service, allowing you to request an image from a whitelisted host server and resize, thumbnail or alter the quality of it.

# 6:18 pm / moddims, apache, imagemagick, images, resizing, thumbnails

2009 » July

MTWTFSS
  12345
6789101112
13141516171819
20212223242526
2728293031