Simon Willison’s Weblog

How Tarsnap uses Amazon Web Services (via) Useful case study, including some thoughts on SimpleDB.

# 14th December 2008, 7:35 pm / simpledb, tarsnap, amazon-web-services, aws, s3, ec2, cloud-computing

Amazon SimpleDB a complete flop? Terry asks if anyone is actually using SimpleDB (related Google searches indicate not, and I’ve personally not heard of anyone using it despite plenty of usage of S3 and EC2). One factor might be that lock-in to EC2 and S3 is pretty small, but if you rely on SimpleDB you’ll need to rewrite your entire application to escape.

# 2nd December 2008, 10:17 am / ec2, s3, amazon-web-services, simpledb, terry-jones, lockin, cloud

Amazon CloudFront. The Amazon CDN front end for S3 has launched. Traffic is 2 cents per GB more than S3. I’d like to see a price comparison with existing CDNs; I have a hunch it’s an order of magnitude less expensive.

# 18th November 2008, 2:37 pm / amazon, cdn, cloudfront, s3

backup_to_s3.py. I wrote Yet Another S3 backup script today. It’s a thin wrapper about boto that doesn’t do anything particularly impressive, but it fits my brain.

# 21st September 2008, 6:51 pm / python, projects, backups, s3, amazon-web-services, sysadmin, boto

We’re Never Content. Amazon will be releasing a proper edge caching CDN on top of S3 “before the end of the year”.

# 18th September 2008, 12:30 pm / cdn, amazon, web-services, s3

Amazon S3 Availability Event: July 20, 2008. Don’t let the newspeak put you off; this is an honest and informative description of the bug that took down S3 last Sunday, although it does include the world’s longest way of saying “we turned it off and on again”.

# 27th July 2008, 5:42 pm / newspeak, amazon, s3, uptime

Django File Uploads (via) Nearly two years in the making, Django’s file upload capacity has received a major (and backwards incompatible) upgrade. Previously, files were uploaded by default in to RAM—now, files larger than 2.5MB are streamed to a temporary file and extensive hooks are provided to customise where they end up—streaming to S3, for example.

# 1st July 2008, 5 pm / django, fileuploads, s3, uploads

Browser Uploads to S3 using HTML POST Forms. I didn’t know you could do this: create a regular HTML form that gives people permission to upload direct to your own S3 bucket, using a signed JSON policy statement in a hidden form field to prevent third parties from abusing your S3 account.

# 27th June 2008, 12:11 pm / s3, amazon, aws, forms, post, json, signing

so-you-wanna-see-an-image (via) WordPress.com use Amazon S3 to store images (presumably to save having to create a massive scalable redundant filesystem themselves) but the images are served via a load balanced memcached / varnishd caching system that they control.

# 1st May 2008, 10:13 am / wordpresscom, caching, amazon-s3, s3, memcached, varnishd

Amazon takes EC2 to the next level with persistent storage volumes. You can store a snapshot of a storage volume to S3 with a single API call, making backups trivial.

# 14th April 2008, 8:04 am / ec2, storage, backups, s3, virtualization, rightscale

Amazon SimpleDB overview. Attribute values are limited to 1,024 bytes; Amazon suggest that you store larger fields in S3 and use SimpleDB to query metadata about those objects.

# 14th December 2007, 11:39 am / simpledb, amazon, web-services, s3, metadata

LastGraph. Now Available. Andrew Godwin has relaunched his LastGraph Last.fm graphing application. The new version is built on Django and S3 and uses Andrew’s Graphication graphing library based on Cairo.

# 15th October 2007, 10:02 pm / graphication, django, andrew-godwin, cairo, lastfm, lastgraph, python, s3, graphing

Amazon Gets an SLA (But I Still Can’t Use It). “Ontario’s Freedom of Information and Protection of Privacy Acts (FIPPA) don’t allow me to store sensitive information (e.g., students’ work) in jurisdictions that permit secret warrants, like those mandated by the USA PATRIOT Act.”

# 9th October 2007, 3 pm / patriotact, privacy, canada, ontario, fippa, gregwilson, s3, amazon, aws

Amazon S3 Service Level Agreement (via) Went in to effect on the 1st of October. Promises 99.9% uptime over a monthly billing cycle or you get “service credits” towards future S3 payments.

# 9th October 2007, 12:52 am / jeffrey-mcmanus, s3, sla, amazon, web-services, uptime, aws

Interview with Leah Culver: The Making of Pownce. Django + Perlbal + S3 + AIR.

# 7th July 2007, 10:05 pm / leah-culver, pownce, django, perlbal, s3, air, python

Processing Web Documents using Alexa Web Search, Amazon S3 and Amazon EC2. I’m not sure when it happened, but Alexa Web Search can be hooked in to EC2 now—presumably with free bandwidth between the two.

# 1st July 2007, 7:19 pm / ec2, alexa, aws, amazon, s3

Mass Video Conversion Using AWS. How to use S3, SQS, EC2, ffmpeg and some Python to bulk convert videos with Amazon Web Services.

# 3rd April 2007, 11:44 pm / aws, amazon, python, s3, sqs, ec2, ffmpeg

boto. Python library for accessing Amazon’s S3, SQS and EC2 Web Services, with excellent documentation.

# 11th February 2007, 12:17 am / python, amazon, s3, sqs, ec2, boto, aws

An S3 AJAX Wiki. Les continues to innovate against S3.

# 22nd April 2006, 7:09 pm / s3, javascript, ajax, wiki, les-orchard, aws

How I’m using Amazon S3 to serve media files. Adrian’s saving server overhead on ChicagoCrime by serving media from S3.

# 7th April 2006, 6:51 pm / adrian-holovaty, s3, amazon, chicagocrime, aws

JavaScript apps with read/write access to S3. JS apps hosted on S3 could read and write to the store.

# 4th April 2006, 9:33 am / javascript, s3, aws

Backing Up Flickr Photos with Amazon S3. 25 lines of Python.

# 22nd March 2006, 6:53 pm / flickr, matt-croydon, amazon, python, s3, aws

BitBucket—Experimenting with Amazon S3 Service in Python (via) Nice pythonic API.

# 22nd March 2006, 6:05 pm / amazon, s3, python, aws

Amazon S3. Game changing.

# 22nd March 2006, 5:59 pm / amazon, s3, aws