Simon Willison’s Weblog

Creating Shazam in Java. Using a Fast Fourier Transformation. # 22nd September 2010, 9:39 pm

ElasticSearch: Your Data, Your Search. A neat example of how ElasticSearch’s schemaless indexes and native JSON support make it ridiculously easy to index different types of data and run queries across them. # 12th February 2010, 3:22 pm

Elastic Search (via) Solr has competition! Like Solr, Elastic Search provides a RESTful JSON HTTP interface to Lucene. The focus here is on distribution, auto-sharding and high availability. It’s even easier to get started with than Solr, partly due to the focus on providing a schema-less document store, but it’s currently missing out on a bunch of useful Solr features (a web interface and faceting are the two that stand out). The high availability features look particularly interesting. UPDATE: I was incorrect, basic faceted queries are already supported. # 11th February 2010, 6:33 pm

twitter-text-conformance (via) This is a neat idea: Twitter have released open source libraries for parsing standard tweet syntax in Ruby and Java, but they’ve also released a set of YAML unit tests aimed at anyone who wants to implement the same parsing logic in other languages. # 6th February 2010, 3:39 pm

Plurk: Instant conversations using Comet (via) Plurk’s comet implementation sounds pretty amazing. They’re using a single quad-core server with 32GB of RAM running 8 Node.js instances to serve long-polled comet to 100,000+ simultaneous users. They switched to Node from Java JBoss/Netty and found the new solution used 10 times less memory. # 1st February 2010, 10:13 am

Frank Wierzbicki: Leaving Sun. Frank performed miracles at Sun and before, helping bring the Jython project out of stasis and turning it in to an active, community maintained modern Python implementation. If you’re looking for an expert Python/Java/Dynamic languages guy you should snap him up. # 4th November 2009, 10:33 pm

Play framework for Java. I’m genuinely impressed by this—it’s a full stack web framework for Java that actually does feel a lot like Django or Rails. Best feature: code changes are automatically detected and reloaded by the development web server, giving you the same save-and-refresh workflow you get in Django (no need to compile and redeploy to try out your latest changes). # 25th October 2009, 11:21 pm

Introducing Cloudera Desktop. It’s a GUI for Hadoop, and under the hood is a whole stack of open source software, including Python, Django, MooTools, Twisted, lxml, CherryPy, Mako, Java and AspectJ. # 21st October 2009, 6:48 pm

And so it goes, around again. Charles Miller on Java, pointing out that if you don’t have closures and first-class functions you end up having to add band-aid solutions and special case syntactic sugar. Python’s lack of multi-line lambdas leads to a similar (though less pronounced) effect. # 3rd September 2009, 9:46 am

Scriptlets—Quick web scripts (via) From the prolific Jeff Lindsay, a pastebin-style tool for short server-side scripts written in Python, JavaScript or PHP that executes them within a Google App Engine powered sandbox. The Java code that implements the service is available on GitHub. # 13th August 2009, 1:51 pm

Jython 2.5.0 Final is out! It’s been a long time coming—congratulations to the team. # 16th June 2009, 11:21 pm

Critical Mac OS X Java Vulnerabilities. There’s a five month old Java arbitrary code execution vulnerability which hasn’t yet been patched by Apple. Disable Java applets in your browser until it’s fixed, or random web pages could execute commands on your machine as your user account. # 19th May 2009, 7:07 pm

oauth-signpost. The Qype API uses OAuth to sign client requests with the developer’s API key, so it’s not surprising to see them release a Java OAuth signing library compatible with Google’s Android mobile platform. # 7th May 2009, 7:33 am

Using Scala with Google App Engine. Scala works, but I haven’t seen confirmation on actors yet (which are likely to break due to their dependency on threads). # 11th April 2009, 3:28 pm

Dynamic languages on Google App Engine—an overview. Ola Bini’s notes on exploring the new Java support for App Engine with the aim of getting JVM dynamic languages such as JRuby running. Restrictions include a complete lack of threads (which will make it hard to get Scala up and running), but JRuby trunk now works without modification. # 8th April 2009, 2:08 pm

Heap Dump Analysis. Using jmap to dump the JVM’s memory to disk, then analysing it using the visualvm GUI tool. # 2nd April 2009, 10:34 am

Magic properties make Firefox synchronously load the Java plugin. Even defining a function called sun() (or several other symbols) will trigger the Java VM to be loaded, dramatically hurting the performance of your page. # 27th February 2009, 4:03 pm

CloudMade: A Summary of the Future of Mapping. CloudMade are now offering commercially supported APIs on top of OpenStreetMap, including geocoding, routing and tile access libraries in Python/Ruby/Java and a very neat theming tool that lets you design your own map styles. This is really going to kick innovation around OpenStreetMap up a notch. # 17th February 2009, 11:25 am

Apache Qpid. A new open source AMQP message queue with implementations in C++ and Java, developed by engineers from Red Hat, IONA and JP Morgan Chase. Anyone tried this yet? Looks pretty good on paper. # 5th February 2009, 11:01 pm

Sloppy—the slow proxy. Java Web Start GUI application which runs a proxy to the site of your choice simulating lower connection speeds—great for testing how well your ajax holds up under poor network conditions. # 13th January 2009, 4:17 pm

“By installing Java, you will be able to experience the power of Java”. Wow, that’s some bad copy. # 12th January 2009, 10:28 pm

How we use IRC at Last.fm. With IRCCat, an elegant Java IRC bot that accepts Twitter-like messages to a network port (generally sent using netcat) and directs them to a user or channel. # 8th January 2009, 7:04 pm

Private Messages with cometD Chat. cometd-java (a Java servlet reference implementation of the Bayeux protocol) can be extended with BayeuxService subclasses that run within the server itself. # 16th October 2008, 2:14 pm

Cascading. A Java API abstraction layer over Hadoop that lets developers think in terms of pipes and filters rather than map/reduce. The Cascading developers claim that this model is easier to understand and less error prone. # 1st October 2008, 1:22 pm

Keyczar (via) New open source cryptography toolkit from Google, designed to get algorithm selection, key rotation and versioning right so you don’t have to. Java and Python versions are available; the Python version depends on PyCrypto. # 13th August 2008, 1:20 pm

Protocol Buffers: Google’s Data Interchange Format. Open sourced today. Highly efficient binary protocol for storing and transmitting structured data between C++, Java and Python. Uses a .proto file describing the data structure which is compiled to classes in those languages for serializing and deserializing. 3-10 times smaller and 20-100 times faster than XML. # 8th July 2008, 8:20 am

3 and 1/2 minutes to sort a Terabyte, and a look at Hadoop’s code structure. Bill de hÓra uses some clever static analysis tools to explore Hadoop’s 100,000+ lines of code. # 7th July 2008, 2:15 pm

Spicing Up Embedded JavaScript. John Resig collects the various ways in which a JavaScript interpreter can be hosted by Python, PHP, Perl, Ruby and Java. There are full JS implementations in PHP, Perl and Java; Ruby and Python both have modules that use an embedded SpiderMonkey. # 15th June 2008, 11:32 am

Multi-Inflection-Point Alert. Dammit, Tim, stop giving away our competitive advantages! # 26th April 2008, 6:48 pm

Transitioning from Java Classes to JavaScript Prototypes. Peter Michaux shows how JavaScript’s prototypal inheritance can run rings around traditional Java-style classes once you figure out how to take advantage of it. # 10th February 2008, 3:10 pm