Sharding Counters on Google App Engine. “While the datastore for App Engine scales to support a huge number of entities it is important to note that you can only expect to update any single entity, or entity-group, about five times a second”. This article explains a technique for sharding writes across multiple counters in detail, including a way to keep a memcache counter updated at the same time for faster reads.
Sign in with 
OpenID
The obvious question is, when those writes start serializing, why doesn't the data layer handle this sharding transparently, or at least provide introspection so that your code can dynamically adjust the amount of sharding?
Jeremy Dunck - 29th January 2009 05:09 - #