Feed Sign in with OpenID OpenID

Simon Willison’s Weblog

Django on Jython (via) Outstanding work from Jim Baker and the Jython team: Django now runs on the modern branch of Jython, with a couple of patches and some failed doctests due to dictionary order (a problem with Django’s test suite).

Tagged , , ,

9 comments

  1. Hashes in test suites are a common problem. It's pernicious as it's usually a case of “works for me” when a problem is reported.

    It's not just tests though. I was battling some Java code yesterday that was effectively untestable due to its forced use of a HashMap (and no way for me to change that to a LinkedHashMap).

    Dominic Mitchell - 4th January 2008 14:33 - #

  2. Footnote: a simple way to get around this when using doctest is to replace:

    >>> foo
    {"key": "value"}

    with

    >>> foo == {"key": "value"}
    True

    (it's not quite as handy if the test actually breaks, but that's usually not a big problem).

    Fredrik - 4th January 2008 23:03 - #

  3. It's not that we're unaware of this problem in Django's tests, but it's very hard to fix without either adding extra code merely to support the tests (with the corresponding and unnecessary performance impact for every real-world use-case) or making the doctests less readable.

    The case in question is where we print out HTML strings and the order of attributes in the HTML element happens to be the natural hashing order. Not printing these strings would defeat the purpose, since the doctests are also acting as documentation of the results. I don't really know what a good solution is here (and I've spent more than a few minutes trying to fix it in the past; it's hardly an unknown issue).

    Malcolm Tredinnick - 6th January 2008 03:20 - #

  4. For HTML attributes, how about a policy of only ever outputting them in alphabetical order? E.g. in http://code.djangoproject.com/browser/django/trunk /django/newforms/util.py#L6 doing this:

    return u''.join(sorted([u' %s="%s"' % (k, escape(v)) for k, v in attrs.items()]))

    Simon Willison - 6th January 2008 09:54 - #

  5. "I don't really know what a good solution is here (and I've spent more than a few minutes trying to fix it in the past; it's hardly an unknown issue)."

    Use strict lexical order. Just loop over sorted(attrib.items()) instead of attrib.items().

    (see xml.dom.minidom, xml.etree.ElementTree etc for prior art)

    Fredrik - 6th January 2008 12:49 - #

  6. I usually avoid the dict doctest problem by doing

    
    >>> from pprint import pprint
    >>> pprint(mydict)
    

    which prints in sorted key order and gives pretty layout, too.

    akaihola - 7th January 2008 22:36 - #

  7. Using strict lexical ordering is exactly what I mean by unnecessary overhead. It's wasted computational cycles merely to satisfy the test suite. The actual use of this code (as interpreted HTML) doesn't require it, so we would be putting in a performance penalty for no useful reason. This is where agile-style testing breaks down: when it adversely impacts your code just to satisfy tests.

    Malcolm Tredinnick - 9th January 2008 04:43 - #

  8. I'd agree with that for large performance penalties, but we're talking about fractions of a fraction of a second here. I benchmarked a sorting v.s. a non-sorting flatatt function using timeit and there was about a 13% difference, but both functions execute in about 0.0001 second so there's no chance they'd ever have a meaningful impact on performance.

    http://simonwillison.net/static/2008/attrs_benchma rk.py

    Simon Willison - 9th January 2008 09:23 - #

  9. What Simon said.

    Saving a fraction of a fraction of a second (Python's sort is fast! Really fast!) by doing the wrong thing is C-style programming, not Python-style ;-)

    (And if you're concerned about cycles, there are *tons* of places in Django that could be made faster with enough micro-tuning. E.g why is render_attrs returning a string instead of a list, for example? Get rid of the top-level join (since you're probably already joining the result with other things somewhere else) and use iteritems instead of items, and you're suddenly able to do the sort for free...)

    Fredrik - 9th January 2008 13:07 - #

Sign in with OpenID

Auto-HTML: Line breaks are preserved; URLs will be converted in to links.

Manual XHTML: Enter your own, valid XHTML. Allowed tags are a, p, blockquote, ul, ol, li, dl, dt, dd, em, strong, dfn, code, q, samp, kbd, var, cite, abbr, acronym, sub, sup, br, pre

A django site