Blogmarks
Filters: Sorted by date
Changes in Opera’s user agent string format (via) How depressing... Opera 10 will ship with 9.80 in the User-Agent string because badly written browser sniffing scripts can’t cope with double digits.
Dice-O-Matic hopper and elevator. An outstanding piece of applied geekery, now generating dice rolls for GamesByEmail.com. “It is a 7 foot tall, 104 pound, dice-eating monster, capable of generating 1.3 million rolls a day.”
You ask, they answer: Neal’s Yard Remedies. After reading the comments, something tells me Neal’s Yard Remedies may be regretting their decision to answer questions from Guardian readers.
geocoders. A fifteen minute project extracted from something else I’m working on—an ultra simple Python API for geocoding a single string against Google, Yahoo! Placemaker, GeoNames and (thanks to Jacob) Yahoo! Geo’s web services.
Testing Django Views for Concurrency Issues. Neat decorator for executing a Django view under high concurrency in your unit tests, to help spot errors caused by database race conditions that should be executed inside a transaction.
The Web vs. the Fallacies. Tim Bray on how the architecture of the Web helps developers handle the Fallacies of Distributed Computing.
uuidd.py. Neat implementation of an ID server from Mike Malone—it serves up incrementing integers over a socket (using Python’s asyncore for fast IO) and records state to a file only after every 10,000 IDs served, so most of the time it’s not reading or writing to disk at all. If the server crashes it doesn’t matter because it can start up again at an integer it’s sure hasn’t been used before.
On the Anonymity of Home/Work Location Pairs. Most people can be uniquely identified by the rough location of their home combined with the rough location of their work. US Census data shows that 5% of people can be uniquely identified by this combination even at just census tract level (1,500 people).
JS-Placemaker—geolocate texts in JavaScript. Chris Heilmann exposed Placemaker to JavaScript (JSONP) using a YQL execute table. Try his examples—I’m impressed that “My name is Jack London, I live in Ontario” returns just Ontario, demonstrating that Placemaker’s NLP is pretty well tuned.
Introducing Yardbird. I absolutely love it—an IRC bot built on top of Twisted that passes incoming messages off to Django code running in a separate thread. Requests and Response objects are used to represent incoming and outgoing messages, and Django’s regex-based URL routing is used to dispatch messages to different handling functions based on their content.
Muck Rack: Links posted by Guardian Journalists on Twitter. I’m rather impressed by the Sawhorse Media collection of Twitter aggregation sites (Muck Rack aggregates journalists)—a simple idea very well executed. Here’s a nice example—this page shows links posted to Twitter by known Guardian journalists, but goes a step further and scrapes in the favicon, the real title of the page and resolves the domain from any shortened links.
Flickr Shapefiles Public Dataset 1.0. Another awesome Geo dataset from the Yahoo! stable—this time it’s Flickr releasing shapefiles (geometrical shapes) for hundreds of thousands of places around the world, under the CC0 license which makes them essentially public domain. The shapes themselves have been crowdsourced from geocoded photos uploaded to Flickr, where users can “correct” the textual location assigned to each photo. Combine this with the GeoPlanet WOE data and you get a huge, free dataset describing the human geography of the world.
Fake Reviews. Now now kids, play nice... Not at all surprised to hear this—nefarious iPhone app developers (in this case the team behind “London Tube”, an inferior version of Malcolm Barclay’s marvellous “Tube Deluxe”) have been caught leaving fake negative reviews on rival applications in the App Store. This is an excellent argument for adding friends/followers or importing an existing social graph—I’d much rather see reviews from people in my social network than strangers who may turn out to be sock puppets.
Dinky pocketbooks with WebKit transforms. Nat used 90 degree CSS transform rotations in print stylesheets for WebKit and Safari to create printable cut-out-and-fold pocketbooks from A4 pages. Very neat.
TwitterAlikeExample—redis. Excellent example of how you design a moderately complex system against a scalable key-value store (in this case redis). Most “how to build Twitter” code examples fail to address the hard problem of scaling user inboxes, but this one tackles it head on.
Working with Python and RabbitMQ. Nathan Borror eliminates the boilerplate needed to talk to RabbitMQ (or any other AMQP queue server) from Python.
AWS Import/Export: Ship Us That Disk! Andrew Tanenbaum said “Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway”, and now you can ship your storage device direct to Amazon and have them load the data in to an S3 bucket for you.
Yahoo! Placemaker. Really exciting new API from Yahoo!—Placemaker accepts a block of text (or a URL to HTML or RSS) and extracts and returns geographical locations mentioned in the text. I just ran my djng blog entry through it and it pulled out “Prague” as the only location mentioned. This should be really useful for adding geodata to existing textual content.
Yahoo! Geo: Announcing GeoPlanet Data. The Yahoo! WhereOnEarth geographic data set is fantastic, but I’ve always felt slightly uncomfortable about building applications against it in case the API went away. That’s not an issue any more—the entire dataset is now available to download and use under a Creative Commons Attribution license. It’s not entirely clear what the attribution requirements are—do you have to put “data from GeoPlanet” on every page or can you get away with just tucking the attribution away in an “about this site” page? UPDATE: The data doesn’t include latitude/longitude or bounding boxes, which severely reduces its utility.
Google Maps Data API (via) I’m disappointed by this one—it’s really just a CRUD store for the KML files used in Google MyMaps. It would be a lot more useful if it let you perform geospatial calculations against your stored map data using some kind of query API—a cloud service alternative to tools like PostGIS.
Offline Processing on App Engine: a Look Ahead. A session at IO next week: “App Engine was designed to run request-driven web applications, although this will change in the coming year with the release of a number of offline computing components. In this session, we’ll explore the task queue/executor model of computation and some of the more interesting applications.”
EC2: Creating an Image. Here’s the easier way of creating your own AMI: start with a running instance in EC2, then customise it to fit your purposes and create a new bundle (and then AMI) using the ec2-bundle-vol command.
HOWTO Building a self-bundling Debian AMI. Not as terrifying as you would have thought. Also contains some neat hints as to how some of the more magical parts of EC2 work (like the way your SSH public key automatically ends up in /root/.ssh/authorized_keys).
Critical Mac OS X Java Vulnerabilities. There’s a five month old Java arbitrary code execution vulnerability which hasn’t yet been patched by Apple. Disable Java applets in your browser until it’s fixed, or random web pages could execute commands on your machine as your user account.
aws—simple access to Amazon EC2 and S3. The best command line client I’ve found for EC2 and S3. “aws put --progress my-bucket-name/large-file.tar.gz large-file.tar.gz” is particularly useful for uploading large files to S3. Written in Perl (with no dependencies), shelling out to curl to do the heavy lifting.
Django tip: Caching and two-phased template rendering. Neat trick for expensive pages which can be mostly cached with the exception of the “logged in as” bit—run them through the template system twice, caching the intermediary generated template.
resty. 58 lines of bash provides a better command-line interface to RESTful APIs, using curl under the hood. This should save me from running “man curl” several times a week.
The Little Manual of API Design (PDF). A concise, highly readable guide to designing APIs that are “Complete, Easy to learn and memorize, lead to readable code, hard to misuse, and easy to extend”, based on lessons learnt over many years of development of the Qt framework.
python-daemon (via) A library for correctly creating Unix daemon processes in Python, implementing the proposed PEP 3143 API.
New Features for EC2: Elastic Load Balancing, Auto Scaling, and Amazon CloudWatch. EC2 now fulfils the promise of “magic scaling in the cloud” out of the box—CloudWatch monitors performance of your EC2 instances without needing to install any monitoring software, Auto Scaling allows you to configure “scaling triggers” which start up new instances based on information from CloudWatch, and Elastic Load Balancing balances requests across all available instances.