What can be going on with your custom management commands in Django 1.10

December 25, 2016

If you update a relatively old Django project from 1.9 to 1.10 or later and you have added your own custom management commands that take positional arguments, it's possible that those commands will abruptly stop working (well, stop accepting positional arguments). Also, although it is not explicitly documented, the *args parameter of your Command's handle() function is now often completely meaningless and you won't see anything passed in it (although when this happens is fairly obscure).

If you were still using the optparse based approach to argument parsing in your custom management commands, this is sort of expected; the 1.10 release notes mention, in the large 'features removed in 1.10' section, that 'support for optparse is dropped for custom management commands' (and you'll have been getting deprecation warnings about that in 1.9). However this happens even if you had already switched over to using argparse based argument parsing (ie, your custom management command class has an add_arguments function). Your code worked fine in Django 1.9 and failed in Django 1.10, despite no deprecation warning.

So, here is what is going on. In Django 1.9 and earlier, code in django.core.management.base's BaseCommand class hierarchy silently introspected your command class to see if it had an args member. If it did and you were already using argparse, BaseCommand silently added an extra args argument to your argument parser that swept up all positional arguments:

if not self.use_argparse:
   ... old optparse code ...
   if self.args:
      # Keep compatibility and always accept
      # positional arguments, like optparse when
      # args is set
      parser.add_argument('args', nargs='*')

Later, other code in BaseCommand would silently take the value of the args argument from argparse's results and turn it into the args parameter for your handle() function (removing it from the argparse results in the process):

# Move positional args out of options to
# mimic legacy optparse
args = cmd_options.pop('args', ())

Note that this specifically happened when you had switched over to using argparse. That this catch-all argument was added (without real documentation) meant that in Django 1.9, if you still had an args member, you could not add your own argument to collect some or all of the positional arguments because it would clash with the automatically added argument here. And really confusing things might happen if you called some argparse argument 'args', because it would be stolen as the args parameter for your handle() function.

In Django 1.10, the first chunk of this code was changed so that an extra args argument is no longer automatically added to your argument parser if you still had an args member in your command class. Now such custom management commands could not get their extra positional arguments (and I believe would fail with a 'cannot parse command line arguments' error if you tried to supply some). However, watch out, because the second chunk of code is still there. If you have an argparse argument called 'args', Django will silently remove it from the argparse result and pass it to your handle() command as the args parameter.

(Of course this is kind of a feature. As it stands now, you can just add your own copy of the old Django 1.9 parser.add_argument call and nothing else in your code has to change. But it smells like a hack to me and I wouldn't be surprised if Django made this magic behavior disappear at some point.)

In my opinion, this automatic addition of an args argument should have been explicitly deprecated, meaning both a deprecation warning in Django 1.9 and an explicit mention in Django 1.10's documentation. Probably the magic behavior of converting an args argparse argument in the handle() args parameter should have been deprecated at the same time, but I don't have strong opinions.

(This is the kind of entry I write because we stumbled over this and then I went and dug the details out of the Django revision history, so I'm certainly not going to waste all that work.)

Written on 25 December 2016.
« I should stop reading some mailing lists during breaks and vacations
Some new-to-me Vim motion commands that I want to try to remember »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Dec 25 01:59:52 2016
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.