Some bits on making Python's argparse module work like Unix usually does

August 22, 2020

I recently discovered that argparse allows you to abbreviate long options, and then Chris Wellons wrote about Conventions for (Unix) Command Line Options, which included a criticism of argparse. I'm not going to write about how to make argparse behave today, because I haven't explored that in full; instead, this is some quick notes from the documentation and my past experiences.

First, both Wellons and I had a bad reaction to argparse accepting abbreviated options. However, based on the documentation you probably have to accept it, because of an important postscript note:

Changed in version 3.8: In previous versions, allow_abbrev [being False] also disabled grouping of short flags such as -vv to mean -v -v.

Almost no one has Python 3.8, which means that the cure here is worse than the disease. Not accepting grouped short flags is much worse than accepting abbreviated long flags, so until Python 3.8+ is pervasive we're stuck with the latter.

As it implicitly documents (in the form of its example), argparse allows the non-traditional style of options being intermixed with non-option arguments on the command line, instead of requiring all options to be before non-options. There is no way to control this. Argparse does accept '--' to terminate the option list (with some caveats), after which things that look like options are non-option arguments.

In general, using the nargs argument for add_argument() is neither necessary nor useful for options (and has issues when used with non-options). Setting things like 'type=str' or 'action="append" causes the argument parser to do the right thing; similarly, it does the right thing when the action doesn't consume any argument value (this behavior is documented in a postscript of the nargs section). As Wellons noted, argparse can fall down badly if you attempt to create an option that takes an optional value. Fortunately, I don't think you should do that and should stick to options either always taking a value or never doing so. Argparse's own examples use 'nargs="?"' for non-options arguments that are in fact optional.

Argparse makes a weird attempt to handle command line arguments that are negative numbers, as documented in Arguments containing -. This isn't how traditional Unix commands behave with such arguments, where a leading '-' is a leading '-' no matter what, with no attempts to guess at what should happen. This behavior is not currently optional and I don't think there's a really good way to trick argparse into not doing it.

(Actually reading much of the argparse documentation has already taught me useful things I didn't know, such as how the dest argument is optional. I'm not sure I'd want to ever leave it out, though; explicit is better than implicit, and using 'dest' leaves a visible reminder in the code of what attribute holds the result.)


Comments on this page:

By Eugene at 2022-02-19 01:22:05:

While I am not a big fan of argparse (for entirely different reasons) your criticism of its behavior w.r.t. "-1" is misplaced. See https://pubs.opengroup.org/onlinepubs/009696799/basedefs/xbd_chap12.html for Posix guidelines on argument syntax, specifically item 6

Unless otherwise specified, whenever an operand or option-argument is, or contains, a numeric value:

  • The number is interpreted as a decimal integer. [...]

So argparse is following the Unix standard here. (It actually goes a bit further than standard requires and recognizes things like "-2.5" as numbers too) Once you think about it a bit it makes sense. When a parser encounters "-1" without explicit option "-1" being defined it needs to make a choice: is this user attempt to specify an option (and so it will error with "invalid option -1") or an argument "negative 1". The argument could be positional or an argument to an option - the logic is the same.

Second ability to abbreviate long options has a history that goes to the very beginning of long options. Both Gnu and BSD getopt_long allow this. You can argue that this was a bad decision back then but blaming Python parser for following the de-facto standard is quite unfair.

Thirdly, the original article you were reacting to is plain silly. The guy claims that "--" doesn't work in argparse - which is false. It does work. The reason his example errored out is that he forgot to define a positional argument.

There are many things to criticize in Python argparse: ability to have more than 1 argument to an option (this is plain evil), bloated interface, lack of good support for sequential parsing and more but the issues you raise aren't among them.

Written on 22 August 2020.
« Link: Why Did Mozilla Remove XUL Add-ons?
The Linux kernel bugzilla (and others) get spammed (of course) »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat Aug 22 23:59:39 2020
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.