Unix shells and the problem of too-smart autocompletion

November 18, 2016

Unix shells have been doing basic file-based autocompletion for decades, but of course basic filename and command completion isn't enough. Modern Unix shells increasingly come with support for intelligent autocompletion that gets programmed to be command and context sensitive for various commands. A modern shell will not just complete filenames for less, it will (perhaps) complete hostnames for ssh commands, automatically know what arguments some command options take, and so on. The degree that it can do this is limited only by the ingenuity and diligence of the crowd of people who write all the autocompletion rules.

All of this is great when it works, but the problem with smart autocompletion is that it can be too smart, so smart that it wraps back around to 'non-functional'. When shells allowed command specific smart autocompletion, they necessarily allowed people to break autocompletion for some or all of a command. This is unfortunately very frustrating to experience; from the user perspective your conveniently smart shell suddenly reverted to not even having filename completion for no readily apparent reason. When you're used to it, losing even filename completion is really irritating.

(If you are not really aware of the fine details of how command line completion works, this may be an extremely hard to understand experience. From your perspective, your shell works some of the time and fails some of the time and it may seem random as to just why.)

You might think that such bugs would be promptly stamped out. Well, it apparently depends on just what the problem is. If there's an outright bug in the autocompletion rules (or code) for a command, something that spits out errors and large-scale failures, it will indeed probably get fixed fairly rapidly. But the more subtle way to do this is to simply have the autocompletion rules just not handle some of the command's command line options; if such an option is used, the rules just bail out and its argument gets no autocompletion. If the option is relatively infrequently used, this issue can remain unfixed for years.

In related news, people using Bash and the reasonably popular bash-completion package of Bash command-specific completions cannot normally complete the very long filename for the PKCS#11 shared library that you need to give to 'ssh-add -e' and 'ssh-add -s'. As far as I can tell, this issue has been present since 2012 when the ssh-add completion was added. Probably you're going to write a script anyways, just because who wants to even autocomplete the full path to shared libraries on any sort of routine basis.

(I believe the issue is that the completion function simply bails out when it's completing something following a -t, -s, or -e argument.)

PS: It turns out that in Bash you can force a filename-based completion to be done with M-/. I'm not sure I'll remember this the next time I need it (since TAB almost always works), but the Bash people did think ahead here.

PPS: It does kind of impress me that you can write sophisticated autocompletion functions in Bash code and have them run fast enough to make the whole thing work well. Computers really have gotten powerful over the years, which is honestly great to see..

Comments on this page:

At least in my zsh, you can also affix a wildcard to the end of the name then press tab.

By Opk at 2016-11-18 08:06:35:

I can only speak for the zsh autocompletion but following many years of experience with this, not breaking file completion is the number one golden rule for zsh autocompletions. A common case is for commands that take subcommands. New contributors will nearly always have their function fall back to printing a message about an unknown subcommand instead of just doing filename completion.

But while I do confirm that the problem you describe is very real, I find the benefits of shell autocompletion far outweigh such problems. I have a key bound for explicit file completion but it's very rare that I need to use it.

In the case of ssh-add, PKCS libraries are a rarely used feature of a widely used program. Writing autocompletion rules is a great way to learn how to use a program but you can find yourself wondering just what form something like a PKCS library takes. It isn't necessarily clear that it wants a path to a shared library.

From early on in zsh the decision was to include the functions with zsh itself. So they're maintained by people who know zsh well but don't necessarily know the program well. There's an increasing trend for programs to include autocompletion functions with their own sources. I'm not yet sure whether this will improve things or make it worse.

By theamk at 2016-11-18 18:12:58:

I have tried to use bash-completion package that was installed on Ubuntu by default a couple of times, and each time I discovered that it does not support the specific command lines I am used to (I think the last time I tried, it was sudo sh -c ... )

Written on 18 November 2016.
« Link: Twice the bits, twice the trouble: vulnerabilities induced by migrating to 64-bit platforms
Why I don't think subscription-based charging gets you stability »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Nov 18 01:49:02 2016
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.