Simon Willison’s Weblog

106 items tagged “json”

Elastic Search (via) Solr has competition! Like Solr, Elastic Search provides a RESTful JSON HTTP interface to Lucene. The focus here is on distribution, auto-sharding and high availability. It’s even easier to get started with than Solr, partly due to the focus on providing a schema-less document store, but it’s currently missing out on a bunch of useful Solr features (a web interface and faceting are the two that stand out). The high availability features look particularly interesting. UPDATE: I was incorrect, basic faceted queries are already supported. # 11th February 2010, 6:33 pm

jsondns. A JSONP API for making DNS queries, with a nice URL structure. # 30th December 2009, 5:37 pm

Orderly JSON. Essentially the JSON equivalent of RelaxNG’s compact syntax—a pleasant mini-language for describing JSON objects which compiles to the more verbose JSONSchema format. # 23rd December 2009, 2:44 pm

Djangopeople JSON parser. Awesome—Andy McKay has compensated for the lack of an official DjangoPeople API by creating a JSONP screen scraped API and hosting it on App Engine. As far as I’m concerned this is an officially supported feature—I’ll make sure future site changes don’t break it, and when I do add an API I’ll try to keep it compatible and help Andy set up redirects. # 28th November 2009, 11:29 am

Introducing BERT and BERT-RPC. Justification for inventing a brand new serialisation protocol: Thrift and Protocol Buffers both use IDLs and code generation, XML “is not convertible to a simple unambiguous data structure in any language I’ve ever used” and JSON lacks support for unencoded binary data. The result is BERT—Binary ERlang Term—which extracts a format from Erlang in much the same way that JSON extracted one from JavaScript. # 21st October 2009, 10:11 pm

cloud-crowd. New parallel processing worker/job queue system with a strikingly elegant architecture. The central server is an HTTP server that manages job requests, which are farmed out to a number of node HTTP servers which fork off worker processes to do the work. All communication is webhook-style JSON, and the servers are implemented in Sinatra and Thin using a tiny amount of code. The web-based monitoring interface is simply beautiful, using canvas to display graphs showing the system’s overall activity. # 21st September 2009, 11:09 pm

“MongoDB is fantastic for logging”. Sounds tempting... high performance inserts, JSON structured records and capped collections if you only want to keep the past X entries. If you care about older historic data but still want to preserve space you could run periodic jobs to roll up log entries in to summarised records. It shouldn’t be too hard to write a command-line script that hooks in to Apache’s logging directive and writes records to MongoDB. # 26th August 2009, 7:09 pm

Tile Drawer (via) The most inspired use of EC2 I’ve seen yet: center a map on an area, pick a Cascadenik stylesheet URL (or write and link to your own) and Tile Drawer gives you an Amazon EC2 AMI and a short JSON snippet. Launch the AMI with the JSON as the “user data” parameter and you get your own OpenStreetMap tile rendering server, which self-configures on startup and starts rendering and serving tiles using your custom design. # 26th August 2009, 9:32 am

Announcing Alice and Wonderland. Continuing the RabbitMQ “stuff to do with rabbits” naming convention, Alice is a RESTful interface to RabbitMQ which exposes information about vhosts/queues/users/exchanges/etc as JSON. Wonderland is a web UI for RabbitMQ implemented as a pure Ajax application which calls Alice. # 17th July 2009, 9:12 am

MongoDB. Lots of discussions about this at EuroPython today—it’s a document database, very similar to CouchDB but significantly faster and suggested for production use. Best of all, trying it out on OS X is as easy as extracting the tarball and running “bin/mongod --dbpath /tmp/test-mongo-db run”. # 30th June 2009, 7:13 pm

Firefox 3.5 for developers. It’s out today, and the feature list is huge. Highlights include HTML 5 drag ’n’ drop, audio and video elements, offline resources, downloadable fonts, text-shadow, CSS transforms with -moz-transform, localStorage, geolocation, web workers, trackpad swipe events, native JSON, cross-site HTTP requests, text API for canvas, defer attribute for the script element and TraceMonkey for better JS performance! # 30th June 2009, 6:08 pm

disturbyte’s zenqueue. Simple, tiny and fast Python message queue server built on top of coroutines and Eventlet, using JSON over TCP as the message format. I’m impressed with how potentially useful this looks considering the small amount of code. The author benchmarks it at 28 thousand messages/second. # 11th May 2009, 1:27 pm

With YQL Execute, the Internet becomes your database. This is nuts (in a good way). Yahoo!’s intriguing universal SQL-style XML/JSONP web service interface now supports JavaScript as a kind of stored procedure language, meaning you can use JavaScript and E4X to screen-scrape web pages, then query the results with YQL. # 29th April 2009, 10:50 pm

Building Fast Client-side Searches. Flickr now lazily loads your entire contact list in to memory for auto-completion. Extensive benchmarking found that a control character delimited string was the fastest option for shipping thousands of contacts around as quickly as possible. # 19th March 2009, 3:35 pm

A few notes on the Guardian Open Platform

This morning we launched the Guardian Open Platform at a well attended event in our new offices in Kings Place. This is one of the main projects I’ve been helping out with since joining the Guardian last year, and it’s fantastic to finally have it out in the open.

[... 839 words]

Combine JSONP and jQuery to quickly build powerful mashups. jQuery’s JSONP support is one of my favourite little-known features of the library. # 3rd March 2009, 3:17 pm

JsonML (JSON Markup Language). An almost non-lossy serialization format for sending XML as JSON (plain text in between elements is ignored). Uses the (element-name, attribute-dictionary, list-of-children) tuple format, which sadly means many common cases end up taking more bytes than the original XML. Still an improvement on serializations that behave differently when a list of children has only one item in it. # 10th February 2009, 3:03 pm

Open in Browser Firefox Add-on (via) Solves the “application/json wants to download” problem, among others. # 9th February 2009, 10:24 pm

Pragmatism, purity and JSON content types

I started a conversation about this on Twitter the other day, but Twitter is a horrible place to have an archived discussion so I’m going to try again here.

[... 555 words]

husk.org. a flickr machine tag browser (via) Flickr recently added API methods for exploring the machine tags used by the community. Paul Mison has built a neat OS X Finder style interface for exploring them, using JSONP and jQuery. # 15th December 2008, 11:24 pm

YQL—converting the web to JSON with mock SQL. YQL just got a whole lot more interesting to me—I had no idea they were exposing an HTML and RSS scraping tool over a JSONP API in addition to all of the Yahoo! web service methods. # 13th December 2008, 9:39 am

Code your own election mashup with Google’s JSON data. The data that powered Google’s US election results map is available to download as a bunch of JSON files. # 6th November 2008, 8:24 pm

typeface.js. Outstanding hack—renders custom fonts using VML in IE and canvas in everything else, using fonts that are defined as a set of vector paths stored using JSON. # 27th October 2008, 11:45 pm

CSSHttpRequest (via) Devious cross-domain Ajax hack that uses CSS for transport (@import rules with data URIs, but it still works in IE). Similar to JSONP but safer, since JSONP can cause arbitrary JavaScript to execute. # 23rd October 2008, 6:25 pm

FriendFeed launch a real-time API. This is huge: JSONP plus long polling Comet, with “everything since X” tokens to ensure you don’t miss anything. This is the first open Comet API I’ve seen anywhere. Combine this with FriendFeed’s regular API (which allows arbitrary message posting) and you’ve got a really powerful tool for hackers who want to experiment with Comet without rigging up their own infrastructure. # 22nd October 2008, 2:18 pm

Page Inlink Analyzer (via) Here’s why I’m so keen on JSONP APIs—Eric Miraglia’s tool fires off dozens of cross-domain JSON requests to pull together information about inbound links to your site from Yahoo! Site Explorer and del.icio.us. I imagine it would have been uneconomic for him to provide the tool if it had to proxy every request through his own server. # 15th October 2008, 5:23 pm

What’s New in Python 2.6 (via) Python 2.6 final has been released (the last 2.x version before 3.0). multiprocessing and simplejson (as json) are now in the standard library, any backwards compatible 3.0 features have been added and the official docs are now powered by Sphinx (used by Django 1.0 as well). There’s plenty more. # 2nd October 2008, 11:47 am

simplejson 2.0.1. Python’s simplejson JSON library got a whole lot faster while I wasn’t looking. # 1st October 2008, 10:55 pm

addSizes.js: Snazzy automatic link file-size generation. Posted to Nat’s snazzy new blog: a script that uses my json-head API to grab the file size of linked documents on a page and insert those sizes in to the document. # 30th August 2008, 10:39 am

json-tinyurl. Because sometimes you want to be able to create a shorter version of a URL directly from JavaScript without hosting your own server-side proxy. # 27th August 2008, 10:58 am