Simon Willison’s Weblog

Decorator to limit request rates to individual views. Neat piece of code for public facing web APIs written in Django. Update: some smart criticisms in the comments.

Posted 24th September 2008 at 1:13 pm

Tagged apis, decorators, django, python, ratelimiting

2 comments

It's not going to be terribly accurate under load if the ORM is issuing statements like:

UPDATE RequestRate SET value = 382.402

Since under heavy load, you'll have multiple request per second and the value will be whoever got put in last. The snippet should issue a plain SQL query to lower it like so:

UPDATE RequestRate SET value = value - 4.37

This snippet only works semi-reliably for lightly loaded sites.

On a side note, you should really fix your OpenID here. For being such an OpenID advocate, it seems a little ironic that the OpenID implementation here sucks, ie:

1) My original place wasn't saved, so when I got redirected back, I had to go and find the post I was at before I started the whole process (iccckkkk!)

2) It didn't use simplereg to at least ask to bring along my full name, so I had to go and type it... which made me wonder why I bothered using OpenID in the first place.

Ben Bangert - 24th September 2008 18:12 - #
Using the database for this sort of thing is kind of silly. It's not relational data, and it's probably not a huge deal if it's lost (someone may be able to go over your rate limit... owells). The concept isn't bad, but it would probably be a better idea to use a decrementing counter in memcached.

Suppose your 'limit' is 50 requests in 5 minutes, for example. The algo would go something like this:
```
key = create_key(url, remote_ip)
try:
  c = cache.decr(key, 1)
catch ValueError:
  # key not in cache
  cache.set(key, c, 300)
if c <= 0:
  rate_limit_this_user()
```
Mike Malone - 24th September 2008 18:32 - #