Simon Willison’s Weblog

Gearman protocol. Notes compiled by Eric Day, principle author of the C server implementation.

# 13th January 2009, 4:44 pm / gearman, protocol, eric-day

New Gearman Server & Library in C, MySQL UDFs. Gearman, the job queue written for LiveJournal and now used by Digg and Yahoo!, has been rewritten in C. Looks like a good candidate for an easily configured lightweight message queue. Also includes hooks for writing MySQL functions that can interact with queues.

# 13th January 2009, 4:41 pm / mysql, message-queues, digg, gearman, livejournal, queues, scaling, yahoo, eric-day

Techniques for safely consuming external HTTP on demand? I asked this question on programming.reddit.com yesterday and got some really insightful answers, including Joe Stump from Digg describing how Digg Images uses Danga’s Gearman worker queue.

# 15th December 2007, 12:29 pm / http, queue, workers, gearman, reddit, askreddit, joe-stump, digg, danga, scaling