82 posts tagged “amazon-web-services”
2009
AWS Import/Export: Ship Us That Disk! Andrew Tanenbaum said “Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway”, and now you can ship your storage device direct to Amazon and have them load the data in to an S3 bucket for you.
aws—simple access to Amazon EC2 and S3. The best command line client I’ve found for EC2 and S3. “aws put --progress my-bucket-name/large-file.tar.gz large-file.tar.gz” is particularly useful for uploading large files to S3. Written in Perl (with no dependencies), shelling out to curl to do the heavy lifting.
Finding similar items with Amazon Elastic MapReduce, Python, and Hadoop streaming. Tutorial for running Hadoop jobs on Elastic MapReduce using Python and the 2005 Audioscrobbler dataset.
Amazon Elastic MapReduce (via) Hadoop as a service. Basically a web based GUI around Hadoop—you could roll this yourself on EC2 but for a small markup on regular EC2 prices you get to avoid the extra work setting everything up. Data processing scripts can be written in Java, Ruby, Perl, Python, PHP, R, or C++ and are loaded in to S3 before firing off the job.
Manage Amazon EC2 With New Web-Based AWS Management Console. Finally! I’m amazed it took Amazon so long to do this. Managing EC2 instances from a custom Firefox extension was pretty bizarre. It’s a very nice interface, built on top of YUI. Unfortunately you still have to manage your entire virtual server farm using a single shared Amazon account.
2008
How Tarsnap uses Amazon Web Services (via) Useful case study, including some thoughts on SimpleDB.
Amazon SimpleDB a complete flop? Terry asks if anyone is actually using SimpleDB (related Google searches indicate not, and I’ve personally not heard of anyone using it despite plenty of usage of S3 and EC2). One factor might be that lock-in to EC2 and S3 is pretty small, but if you rely on SimpleDB you’ll need to rewrite your entire application to escape.
Trying out Windows on EC2. Phillip Pearson provides the missing documentation.
backup_to_s3.py. I wrote Yet Another S3 backup script today. It’s a thin wrapper about boto that doesn’t do anything particularly impressive, but it fits my brain.
Persistent Django on Amazon EC2 and EBS—the easy way. Useful tutorial on getting Django up and running on EC2 with EBS for a persistent PostgreSQL database.
Browser Uploads to S3 using HTML POST Forms. I didn’t know you could do this: create a regular HTML form that gives people permission to upload direct to your own S3 bucket, using a signed JSON policy statement in a hidden form field to prevent third parties from abusing your S3 account.
2007
Amazon Gets an SLA (But I Still Can’t Use It). “Ontario’s Freedom of Information and Protection of Privacy Acts (FIPPA) don’t allow me to store sensitive information (e.g., students’ work) in jurisdictions that permit secret warrants, like those mandated by the USA PATRIOT Act.”
Amazon S3 Service Level Agreement (via) Went in to effect on the 1st of October. Promises 99.9% uptime over a monthly billing cycle or you get “service credits” towards future S3 payments.
Processing Web Documents using Alexa Web Search, Amazon S3 and Amazon EC2. I’m not sure when it happened, but Alexa Web Search can be hooked in to EC2 now—presumably with free bandwidth between the two.
Mass Video Conversion Using AWS. How to use S3, SQS, EC2, ffmpeg and some Python to bulk convert videos with Amazon Web Services.
boto. Python library for accessing Amazon’s S3, SQS and EC2 Web Services, with excellent documentation.
2006
An S3 AJAX Wiki. Les continues to innovate against S3.
How I’m using Amazon S3 to serve media files. Adrian’s saving server overhead on ChicagoCrime by serving media from S3.
JavaScript apps with read/write access to S3. JS apps hosted on S3 could read and write to the store.
Backing Up Flickr Photos with Amazon S3. 25 lines of Python.
BitBucket—Experimenting with Amazon S3 Service in Python (via) Nice pythonic API.
Amazon S3. Game changing.