optfunc. Command line parsing libraries in Python such as optparse frustrate me because I can never remember how to use them without consulting the manual. optfunc is a new experimental interface to optparse which works by introspecting a function definition (including its arguments and their default values) and using that to construct a command line argument parser. Feedback and suggestions welcome!
Sign in with 
OpenID
I've had the same problem in the past, so I threw together a declarative class-based framework that works much like Django's model and form declarations. I guess I just assumed I was the only one who found optparse hard to deal with, so I didn't bother cleaning it up and releasing it, but now that I see your work, maybe I'll do just that.
For simple setups, though, I think you've actually hit on a fantastic idea here. The naysayers will likely point out that you're missing support for some other optparse features like shorter aliases for common options, but I think stripping it down for the sake of simplicity is great for the many situations where you don't need all the flexibility.
Covering the 80% of cases is exactly what I'm going for. Actually I've got the shorter aliases in there already - it automatically picks a short alias using the first letter of the longer option, falling back on the second letter if the first has already been taken and so on.
If you want finer grained control over the letter, you can define your argument like this:
def foo(bar, q_myoption=True)
Then the short argument will be -q but the long argument will be --myoption
I'm not too worried about other optparse stuff, since it's using optparse under the hood. I'll probably add a documented method for getting back the automatically created OptionParser instance so you can customise it further if you really need to.
I've updated the readme to document the way short and long options are derived from function parameters.
This sounds exactly like what Perl 6 is doing, fyi:
http://perlgeek.de/blog-en/perl-5-to-6/14-main-sub .html
Keith - 28th May 2009 20:50 - #
Ah, okay. I admit, I hadn't read the code yet, just the example, but it certainly looks like you're on the right track. I think **kwargs could be handy, though, if you're writing a wrapper around some other app, and you'd like to pass through arbitrary options that your wrapper doesn't really care about. Like maybe a wrapper around some ID3 library that lets you specify tags right on the command line (id3.py dj_*.mp3 TPE1="Django Reinhardt"). Sure, the use cases might be fewer, but if it can be implemented without compromising the simplicity of the app, I'd go for it.
For ages, I've used my brother's CMDSyntax library when I've wanted to do option parsing:
http://www.boddie.org.uk/david/Projects/Python/CMD Syntax/index.html
Of course, it isn't wrapping functions automatically - you still have to write syntax definitions - but I imagine that you could go from function signatures to syntax definitions, then have some forwarding of the program arguments to those functions.
Personally, the act of writing the definitions makes me think more about the interface to the program than I otherwise would do, and I think it makes me try harder to make more usable command line programs.
Paul Boddie - 28th May 2009 21:53 - #
Hmmm.
Like you I don't remember optparse syntax (I have a cheatsheet for that). Optparse does have a few good things going for it. It supports what I call "composite scripts" (wrapping other scripts into a big command line dispatcher, sort of similar to what manage.py does in django) really easily. It's included in the stdlib. That's a big one, because one could argue that successors to optparse are better (perhaps yours could be included in this list), but for a command line script I usually want to limit dependencies. OptionParser also supports the "version" argument, which is nice (and probably would be easy for you to support).
I prefer optparse's help parameters to your decorators. If I can't remember optparse, I probably won't be able to remember (what seems like more) syntactic sugar for your version. How do you plan on supporting list items as parameters? Say I want to work on a bunch of files for input. Optparse is pretty bad here and I usually resort to comma delimiting the option which seems silly.
One more thing that I would like that no one has supported (yet?) is somehow chaining options to use a .ini file or env variables if they are set. Obviously one problem with that is that everyone prefers their config in their favorite format....
Shameless plug, I mention some of these things in my pycon talk... http://us.pycon.org/2009/conference/schedule/event /61/
matt harrison - 28th May 2009 22:59 - #
This is really great solution for a recurring problem - and right in time for the Django Dash this weekend. (Our app will use a custom daemon in the backend.) Thanks for sharing!
Chris Wanstrath - 29th May 2009 00:06 - #
matt: optfunc is just optparse under the hood, and I plan to make it easier to intercept the OptionParser it creates so anything custom can be done against that.
I might try and compress optfunc.py down to the point where you wouldn't mind just pasting it straight in at the top of a script to solve the dependency issue.
I've got a plan for composite scripts, thanks to @nasrat on Twitter. I think I can solve the list of items / bunch of files issue as well, while still keeping the API completely memorable. Unfortunately I don't have a better alternative to the @arghelp decorator.
Very nice work. I made a couple of fixes and added support for running classes too:
http://github.com/rybesh/optfunc/tree/master
Ryan Shaw - 29th May 2009 01:15 - #
Thanks Ryan, applied.
I've added support for composite scripts / subcommands now - just pass a list of functions to optfunc.run() instead of a single function and the function names will be used to dispatch based on the first argument. It breaks help at the moment though - any tips on getting optparse (the underlying library) to behave well with subcommands would be really useful.
There's a module called argparse that is supposedly better than optparse, but mostly compatible. You might be interested in evaluating it.
In addition to run(), you could add main() which would introspect the Python stack to see if the caller module is __main__ or not. You could then dispense with the "if __name__ == '__main__':" in the caller, saying instead just "optfunc.main(foo)".
Anonymous Coward - 29th May 2009 13:38 - #
I spotted argparse today - it looks like it solves the subcommand problem for one. If it's mostly compatible it may be possible to use it if available and fall back on optparse otherwise.
I like the main idea - I'll look in to that.
I think you can solve the subcommand problem by using two OptionParser instances. With the first one (global), do disable_interspersed_args() so that it stops argument parsing when it sees the command. The command and its remaining arguments are then available in p.args, and can be fed to the OptionParser instance of the subcommand.
Seems like subcommands could easily be done by having a "master" function, main(*args, ham=foo, spam=bar, ...) that would receive and define the global options that apply to each subcommand. The main function could then call optfunc.dispatch or something like that. This would also give a natural place for the main docstring.
pv - 29th May 2009 19:15 - #
Ok, implemented it myself:
http://github.com/pv/optfunc/tree/master
pv - 29th May 2009 19:55 - #
A very similar (seemingly identical) option parser has been implemented a few years ago by Laurent Szyster as part of the Allegra package.
Here is the relevant module:
http://laurentszyster.be/blog/anoption/
Istvan Albert - 29th May 2009 21:51 - #
Istvan: that looks like exactly the same idea. I'm surprised I hadn't heard of it before.
I've implemented the name==main trick, and added some other features like magic stdin/stdout/stderr argument names. I've updated the readme file with the details.
I definitely love this idea and got a bit carried away in hacking on it tonight. I went in thinking I was gonna beef up the sub command support and ended up moving quite a bit of stuff around and brought testing up to 100% coverage. Code is up at [1].
Simon: Have you got plans to register this on pypi?
[1] http://github.com/davisp/optfunc/tree/master
Paul J Davis - 31st May 2009 11:26 - #
I suspect most of us have done ad-hoc one-shot variations of this already, so adding your basic optfunc.main API to the standard library (as optparse.simple, perhaps?) sounds like a no-brainer to me. Just check it in already :-)
I'm less impressed by the nostrict/arghelp decorators - I'd probably use custom objects as default values for those options instead (i.e. if the default value is a subclass of a provided base class, it's up to the default object to define how to process the option string).
And argument specific help texts could be moved into (and automatically extracted from) the docstring.
Why "%prog", btw? Wouldn't using an existing Python placeholder syntax be better?
Here's a setup.py for installing it in site-packages. Regards, Luke
from distutils.core import setup
setup(name='optfunc',
version='1.0',
py_modules=['optfunc'],
)
Luke - 2nd June 2009 10:22 - #
Zed Shaw has a module called args.py (part of lamson), but I think I like this better. I'll probably hack the bits I like together.
nelix - 5th July 2009 03:26 - #
It will be great if default values of options have been automagically added to the help text. Something like the following:
@optfunc.main
@optfunc.arghelp('var', 'just a var')
def run(var='test'):
pass
$ test.py -h
...
Options:
-h, --help show this help message and exit
-v VAR, --var=VAR just a var ('test' by default)
Great library, but what about uploading it to PyPI or at least providing a setup.py script so that the package could be properly installed with pip?