Simon Willison’s Weblog

Items tagged python in Mar

Filters: Month: Mar × python ×

VisiData (via) Intriguing tool by Saul Pwanson: VisiData is a command-line “textpunk utility” for browsing and manipulating tabular data. “pip3 install visidata” and then “vd myfile.csv” (or .json or .xls or SQLite orothers) and get an interactive terminal UI for quickly searching through the data, conducting frequency analysis of columns, manipulating it and much more besides. Two tips for if you start playing with it: hit “gq” to exit, and hit “Ctrl+H” to view the help screen. # 18th March 2019, 3:45 am

import-pypi. A devious Python 3 hack which abuses importlib.machinery to add a hook such that any time you type “import modulename” it checks to see if the module is installed and runs “pip install modulename” first if it isn’t. Intended as a joke, but if you habitually fire up temporary virtual environments for exploratory programming like I do this could actually be a neat little time-saver. # 29th March 2018, 10:16 pm

How to use HDF5 files in Python (via) HDF5: the missing manual. A detailed explanation of the HDF5 file format and how to work with it using the h5py module. HDF5 allows you to efficiently store multiple datasets (plus metatdata about them) in a single file and then load data from them without pulling the entire file into memory—kind of like SQLite but without the SQL support and more optimized for working with arrays. # 19th March 2018, 2:55 pm

Trio Tutorial. Trio is a really nice async library for Python—a simpler alternative to asyncio, with some very clean API design. Best of all, the tutorial is fantastic—it provides a very clear explanation of async/await without diving into the intricacies of coroutines. # 17th March 2018, 3:55 pm

What are some hidden features of Python?

Generators and Iterators are pretty amazing. These two tutorials will really open your eyes as to how powerful they can be:

[... 63 words]

The Onion Uses Django, And Why It Matters To Us. The Onion ported their main site from PHP and Drupal to Django in three months with a team of four developers, including a full migration of their archived content. Their developers answer questions about the switch in this thread on the Django sub-reddit. # 25th March 2010, 6:43 pm

Fun with TextMate and PDB. TextMate bookmarks (against lines in a file) are stored as OS X extended attributes, which can be accessed from Python using the xattr module. Here’s a clever piece of code that uses bookmarks to set breakpoints in the command-line pdb debugger. # 23rd March 2010, 9:48 am

The Web Server Benchmarking We Need. Ian Bicking asks for a WSGI benchmark which emphasises error handling over raw performance—can the server keep serving requests if some of them are CPU bound, I/O bound, wedged or cause a segfault? # 17th March 2010, 10:05 am

Automated deployments with Fabric—tips and tricks. “If it’s not in a Fabric fabfile, it’s not deployable”—I’m slowly applying this philosophy to my personal projects. # 16th March 2010, 11:19 am

Installing PIL on Mac OS X Snow Leopard for use in Google App Engine. PIL installation instructions that actually work... the ’export CC=“gcc -arch i386”’ incantation in particular. Make sure you run install using the Python version that the App Engine dev tools are using (I ran “sudo /usr/bin/python2.6 install”). # 15th March 2010, 4:06 pm

Introducing the PyPy 1.2 release. It’s been a long time coming, but 1.2 is the first PyPy release to ship with a Just-in-Time compiler! Performance looks pretty impressive. # 12th March 2010, 11:54 pm

Cache Machine: Automatic caching for your Django models. This is the third new ORM caching layer for Django I’ve seen in the past month! Cache Machine was developed for zamboni, the port of to Django. Caching is enabled using a model mixin class (to hook up some post_delete hooks) and a custom caching manager. Invalidation works by maintaining a “flush list” of dependent cache entries for each object—this is currently stored in memcached and hence has potential race conditions, but a comment in the source code suggests that this could be solved by moving to redis. # 11th March 2010, 7:35 pm

Is johnny-cache for you? “Using Johnny is really adopting a particular caching strategy. This strategy isn’t always a win; it can impact performance negatively”—but for a high percentage of Django sites there’s a very good chance it will be a net bonus. # 2nd March 2010, 11:44 am Johnny Cache. The blog entry announcing Johnny Cache (“a drop-in caching library/framework for Django that will cache all of your querysets forever in a consistent and safe manner”) to the world. # 1st March 2010, 11:48 am

Apparently [unladen-swallow] is already 30% faster than CPython, and this version is being used to run some of the Python code on YouTube.

Ted Leung # 30th March 2009, 10:10 am

ProjectPlan—unladen-swallow. A branch of Python 2.6 aiming to radically improve performance (the target is a 5x improvement), by compiling Python to machine code using LLVM’s JIT engine. I think this is a Google 20% time project (or maybe not, see the comments). An early version without LLVM is already available for download. # 30th March 2009, 10:09 am

Future roadmap for mod_wsgi. mod_wsgi 3.0 isn’t too far off, and will include Python 3.0 support, WSGI application preloading and internal web server redirection (similar to nginx X-Accel-Redirect). Version 4.0 plans a major architectural change that will allow multiple versions of Python to be run from the same Apache. # 19th March 2009, 5:27 pm

Parrot 1.0.0 “Haru Tatsu” Released! Parrot hits 1.0! Anyone know how complete Pynie, the Python implementation on top of Parrot is? # 19th March 2009, 3:17 pm

Load spikes and excessive memory usage in mod_python. “The final answer? Stop using mod_python, use mod_wsgi and run it with daemon mode instead. You will save yourself a lot of headaches by doing so.” # 16th March 2009, 5:26 pm

Concurrence. Exciting: a Python framework for “creating massively concurrent network applications” (the tutorial benchmarks a Hello World web server at over 8,000 requests a second). It’s implemented on top of libevent using pyrex, can run on either Stackless Python or Greenlets from the py library and ships with a WSGI server, an HTTP client and a DBAPI 2.0 compliant MySQL driver. # 15th March 2009, 1:28 pm

Ruby on Rails 2.3 Release Notes. I’m impressed with how thoroughly Rails has embraced Rack (Ruby’s standardised web framework API, inspired by Python’s WSGI). # 15th March 2009, 1:22 pm

A few notes on the Guardian Open Platform

This morning we launched the Guardian Open Platform at a well attended event in our new offices in Kings Place. This is one of the main projects I’ve been helping out with since joining the Guardian last year, and it’s fantastic to finally have it out in the open.

[... 839 words]

It’s time for a change. Jacob Kaplan-Moss is joining Revolution Systems, who will now be offering professional Django support “to companies who need a Django expert on staff, but can’t afford someone full-time.” # 4th March 2009, 10:30 pm

Django snippets: Smart {% if %} template tag. Chris Beaven’s drop-in replacement for Django’s {% if %} tag that adds comparison operators (less than, greater than, not equal etc) while staying backwards compatible with the less able original. I love it. This is one place where I no longer favour Django’s stated philosophy: I think it’s perfectly reasonable to use comparisons in presentation logic, and I’ve found that in my own code the lack of an advanced if tag frequently leads to pure presentation logic sneaking in to my view functions. # 3rd March 2009, 3:03 pm

Django Development with Djblets. The Review Board team have extracted a library of useful Django utilities from their application. The first to be documented are helpers for reducing boilerplate in custom template tags. # 31st March 2008, 1:19 pm

Graphication. Andrew Godwin’s Python graphing library, based on Cairo. Responsible for the very handsome graphs on The Carbon Account. # 30th March 2008, 7:05 pm

Exposing calendar events using iCalendar in Django. A simple abstraction around the vobject Python library. # 30th March 2008, 6:31 pm

xPyUnit: Uniting in Python with XML reporting. Should be just the ticket for integrating Django’s testing framework with Cruise Control. # 27th March 2008, 12:35 pm

Setup mod_wsgi for Django and Shared Hosting. Tutorial by David Cramer; attached are useful comments from mod_wsgi author Graham Dumpleton. # 26th March 2008, 2:42 pm

Djangofriendly (via) Ryan Berg’s attractive new site collecting ratings and reviews for web hosts that support Django. I’m still happily hosted on a bytemark VPS, which isn’t currently listed on the site. # 26th March 2008, 8:47 am