Simon Willison’s Weblog


7 items tagged “memcache”


Building and Scaling a Startup on Rails: 12 Things We Learned the Hard Way. Lessons learned from Posterous. Some good advice in here, in particular “Memcache later: If you memcache first, you will never feel the pain and never learn how bad your database indexes and Rails queries are”. Also recommends using job queues for offline processing of anything that takes more than 200ms.

# 23rd February 2009, 8:28 am / memcache, message-queues, posterous, rails, scaling

Sharding Counters on Google App Engine. “While the datastore for App Engine scales to support a huge number of entities it is important to note that you can only expect to update any single entity, or entity-group, about five times a second”. This article explains a technique for sharding writes across multiple counters in detail, including a way to keep a memcache counter updated at the same time for faster reads.

# 27th January 2009, 8:27 pm / appengine, caching, google, googleappengine, memcache, python, scaling, sharding

MemcacheDB. A server that speaks the memcache protocol but uses Berkeley DB for reliable persistent storage. Speedy: 20,000 writes/second and 60,000+ reads/second. Includes a full replication mechanism (with custom memcache protocol commands) based on Berkeley DB’s.

# 5th January 2009, 12:37 pm / berkeleydb, keyvaluepairs, memcache, memcachedb, replication, scaling



NginxMemcachedModule. nginx can be set up to directly serve a URL from memcache if the corresponding cache key is set, and fall back to a backend application server otherwise. Application servers can then write directly to memcache when content needs to be cached or goes stale.

# 15th December 2007, 1:59 am / caching, memcache, memcached, nginx, scaling

Clever Caching. Instead of invalidating your cache directly, bump a version number on your model (blog entry or whatever) and use that as part of the cache key. This also gives you dynamic etags for free.

# 5th July 2007, 12:56 am / caching, etags, memcache, michaelkoziarski

libketama (via) A consistent hashing algorithm for memcache clients, from the team at

# 20th April 2007, 6:50 am / hashing, lastfm, lesorchard, memcache