Simon Willison’s Weblog

Implementing filesystems in Python

10th December 2003

LUFS-Python provides a relatively simple API for implementing new Linux filesystems in pure Python. You install the package, write a class implementing methods for handling filesystem operations such as creating a directory, opening/reading/writing/closing a file, creating symlinks etc and finally mount your new filesystem with some special arguments to the mount command.

At first glance, this is a bit of a gimmick—why would you want to write your own filesystem in the first place? We’ve been talking about this at work and came up with a few ideas. How about a filesystem where HTML files saved in a certain directory were instantly run through HTMLTidy and converted in to valid XHTML ? Or a custom network filesystem that saves files on a remote server using GnuPG to encrypt them before transfer? How about a read-only filesystem that lets you browse the contents of a MySQL database? Just imagine being able to use tools such as grep and find to search your database. A module that maps someone elses public web server to your own filesystem, making mirroring as easy as running a recursive cp command. A filesystem that updates a swish-e full-text index every time a file is saved to it—years before Microsoft release Longhorn. The possibilities are endless.

Here’s a really fun idea: a filesystem that implements a dynamic website. Instead of using tools like mod_python to dynamically create pages, implement a filesystem that dynamically creates HTML files as they are requested and set up a stock Apache install with the dynamic filesystem as the document root. Then point ProFTPD at it so you can log in via FTP and mess with your content dynamically. We’re thinking about bulding an FTP interface to our new database driven CMS, but we could just build a filesystem interface and point our FTP server straight at it.

I’m sure there are performance and stability issues that make most of the above more trouble than it’s worth, but I think you’ll agree it’s a pretty exciting technology.