Feed Sign in with OpenID OpenID

Simon Willison’s Weblog

Greasemonkey as a lightweight intermediary

In The architecture of intermediation, Jon Udell discusses the need for a mechanism for a high-level tool for adding custom features to web applications. In Jon’s case, he wants to add a private bookmarks feature to del.icio.us. Jon thought about using a web proxy to intercept and modify del.icio.us pages, but ruled it out as too low-level.

Jon, you need Greasemonkey.

The latest release of the swiss army knife of Firefox extensions adds support for cross-domain XMLHttpRequest calls from greasemonkey scripts. What that means is that you can create a user script (a short JavaScript that will be executed whenever your browser loads specific pages) that can then pull extra data in from another server. This new ability is described in the greasemonkey documentation.

I’m using this for my final year project, a decentralised web annotation system that lets you annotate pages, storing your annotations locally and then sharing your public annotations as a feed (similar to the way RSS aggregators work). The trick there is to run a local web server on some port, then have the Greasemonkey user script (eventually a full extension) communicate with that local server to store and retrieve data. I’m using Ruby on Rails’ built in WEBrick server to prototype the service, and it’s working a treat.

This architecture could be easily adapted to add private bookmarks to del.icio.us—or to add any number of cool features to any number of other sites. Here’s another example: Google’s Desktop Search integrates results from your local drive with the search results page on Google. Using greasemonkey and a local web server tied in to OS X Tiger’s Spotlight indexer, you could add this functionality to any search site you wanted to. Just be sure to lock down the web server to only serve requests from localhost, to avoid sharing search results for your data with anyone on the network who can see your machine.

When people asked me what I was excited about at SxSW, one of my answers was Greasemonkey. This kind of stuff is the reason why.

Posted 30th March 2005 at 6:53 pm

This is Greasemonkey as a lightweight intermediary by Simon Willison, posted on 30th March 2005.

Tagged ,

View blog reactions

Next: Enter the hedgehog

Previous: PyCon observations

14 comments

  1. a decentralised web annotation system that lets you annotate pages, storing your annotations locally and then sharing your public annotations as a feed

    Interesting, Simon. I've been thinking about this as well for sometime since I first heard of Annotea (though admittedly haven't got around to actually trying anything with it).

    Is there a reason you'd want to share via a local server though? Why not automatically upload it to a previously existing publicly available server rather than a local one? Then it could just be an extension of your blog, available as another feed as you suggest. I'm interested in seeing what you do with it, but running another server locally really doesn't appeal to me (though I am only one person)

    Derek Featherstone - 30th March 2005 21:46 - #

  2. If this went commercial / public, all you'd need is a hosted web service that could substitute for the local server and keep your data private. (add some encryption to make it really secure.) Anyway, excellent concepts and I can't wait to see what Simon does with them...

    Michael Moncur - 30th March 2005 22:25 - #

  3. Derek: the plan is for the thing to optionally be able to publish feeds via FTP to a configured server and location. Sharing by a local server has the obvious disadvantage that people behind a router won't be visible to the outside world. I want to run a local server for the functionality becase it's such a convenient way of running a web application (zero server install required). There's a precendent for this: Radio Userland and Amphetadesk are both programs that run a local web server on some high port to provide a web application style interface to users.

    Simon Willison - 30th March 2005 22:38 - #

  4. Yummy.

    Jeremy Dunck - 30th March 2005 23:09 - #

  5. Simon: Google's Desktop Search for windows is another program that uses a local web server, at least I think I read that it does...

    Michael Moncur - 31st March 2005 05:26 - #

  6. Interesting. I've been musing along those very same lines ever since I saw WikiAlong. That sort of thing might be the next del.icio.us, if done right. And as to the feeds: Remember Rael's Peerkat? I once thought of tacking on a Gnutella-like protocol to it, so that you could share feeds from behind firewalls. Ah, those were the days, when I wasn't short of ideas... :)

    Rui Carmo - 31st March 2005 10:29 - #

  7. I wrote a web annotation system for my final year project too (at ic.ac.uk).

    It used a Konqueror plugin + KHTML hacks ( this harks back to KDE 3.0).

    So my system had the following features:

    • P2P
    • Digital signatures
    • Trust network (based on 'categories', eg trusting someone on software doesn't mean trusting them on religion). Trust declarations were comments on people.
    • Moderation taking into account your trustiness. (All this was done for anti spam purposes).
    • Impossible to detect for page authors. Third Voice, the most popular web annotation system was screwed because it was accessible via the DOM. So page authors mashed it up. My one modified stuff to the rendering tree in KHTML,
    • Sort-of-kind-of XLink for selecting what content to comment. Allowed fuzzy matching if dom structure changed. (For sub page matching). I wanted to implement image region commenting but didn't.
    • Comments could also be content ratings or eg spelling corrections.
    • Hash of the document was included and then fuzzy matching was used if it was different.
    • My local server was python based.
    • Easy to switch identities.

    A bunch of other stuff. I'd be glad to discuss it, mail me.

    My plans were to expand it to cover things like distributed ad-cutting from recorded TV shows by commenting on where the ad starts, and also to distributed map making by commenting on locations. It would also be possible to do distributed auctions, etc. But I'm lazy...

    rjw - 31st March 2005 12:31 - #

  8. Impatient to see it.
    How does it compare to iMarkup (via http://slate.msn.com/id/2114791/ )?

    Julien Couvreur - 31st March 2005 23:34 - #

  9. There's local annotation to semi-automatic annotation.

    The WonderWeb IST Project (Owl) [Report]:

    Significant results above and beyond the stated objectives of the project have also been achieved, according to project coordinator Professor Ian Horrocks. These include the development of techniques for the semi-automatic annotation of dynamic websites and the investigation of alternative reasoning techniques.

    This has an obvious collaborative tie to semantic web objectives for the working group.

    Also why don't you embed a rich text editing environment to your comments editing pages that checks for well-formedness instead of commenters having to write all this sugary tag stuff to make it look half-readable?

    Amit - 5th April 2005 11:56 - #

  10. Now we also have GreaseMonkIE for Internet Explorer.

    Bart - 7th April 2005 22:54 - #

  11. We (de.lirio.us) just added private bookmarks. The code is still open so hack it up to provide whatever functionality you need. Perl.

    Steve Malllett - 5th June 2005 20:36 - #

  12. I'm working on server-based web annotation for Moodle using a similar AJAX-based approach. In my case, annotations are displayed in the margin alongside highlighted ranges of text. I am very curious to see your code: when I first heard of Greasemonkey I immediately thought of annotation, but I haven't pursued it yet.

    Geof - 15th June 2005 22:59 - #

  13. another two free web proxies http://anon.emigrantas.com http://proxy.emigrantas.com - text based

    Darius - 6th August 2006 22:03 - #

  14. Great tip. I have wanted to the same thing with my bookmarks as not all of them are relevant for the public.

    Share Trading - 26th September 2006 02:23 - #

Comments are closed.

Previously hosted at http://simon.incutio.com/archive/2005/03/30/lightweight

A django site