Your CLI's completion should know what options you've already typed
25 points by hongminhee
25 points by hongminhee
I dislike interfaces that try too hard to guess what I want. Each parameter that goes into the guess is a bit of context that I need to remember if I want to predict the behavior of the UI.
I'd rather understand my interface than get used to it.
The .NET team had a really nice thing for this. I’m not sure if it was ever released, but they had a CLI parser that came with a grammar description and embedded the grammar in a special section of the executable. PowerShell would then pull that out and so could parse your command line exactly as the program would, including understanding dependent arguments.
OpenStep (and Cocoa) have an interesting model where settings are provided as key-value pairs (where the values can be strings, numbers, dates, binary blobs, dictionaries, or arrays). These are stored as user defaults in domains, where each layer could override settings from the layer below. This let you have app-bundled defaults, system-wide settings, and per-user settings all exposed via the same interface. The final layer was the arguments domain. When you start an OpenStep / Cocoa app, it passes argc and argv to NSApplicationMain, which manages the run loop. This also populates the arguments domain in user defaults with things from the command line. A lot of Apple’s first-party CLI tools work this way: their command-line flags are handled via the CoreFoundation user defaults later and you can set default values for them with the defaults tool (which inspects and modifies the persistent user defaults domains). This doesn’t provide any introspection to the shell though and it would be great if they included a schema in the tool’s plist.
I believe that it was the DEC TOPS-20 operating system that did this in the 1970s. Unix has set our expectations so low.
TOPS-20 actually went a step further (IIRC*): the command’s process was launched as soon as you typed its name, and then it used system calls to communicate with the shell (EXEC) while you typed the command line. All subsequent shells I know of parsed the command line themselves (with varying levels of sophistication) and then launched the command.
So the more fundamental way we’re warped by Unix is the assumption that command arguments should be there at all when you launch the process! TOPS-20 didn’t have argc/argv.
* It may be more correct to say that EXEC handed the terminal over to the command, which then used the COMND library to process the user’s typing — I can’t quite remember.
Edit: my footnote is correct, and here’s a tutorial with a good explanation of how it works.
For the record, even DEC's own (Open)VMS decided to offer nearly no completion on DCL. But yes, seeing TOPS-20 in a PDP-10 emulator was mind-blowing to me — I couldn't believe how backwards even many later systems were by comparison.
There’s really no reason we couldn’t do this today. Just have the shell pass a socket to the command and do some IPC. It’s trendy to build three different completion script generators into the command anyway!
Uh-oh, did I just nerd snipe myself?
Two things come to mind:
oops, that wasn't the command I wanted to run. And it takes three seconds to start up the one I typed, so...
great, now we have to rewrite every command to a new standard
It's kinda sad that we're talking about reintroducing a feature from the late 70s and performance is a concern. If you can't load the completion routine that fast, maybe just skip it.
The second bullet was why I pointed out we're already writing commands to support three completion standards. And those are even (mostly) redundant! (Thanks to everyone who's added fish support, btw.)
Is Emacs’s transient UI the closest we’ve come to this on Unix?
I wouldn't have described transient as being "on Unix", but rather "in Emacs". Doesn't it only work with elisp code, not commands in general?
I guess we’re having different things in mind.
Are there Unix shells that approximate TOPS-20’s completion behavior? I can only think of transient which, yes, isn’t a shell nor an extension for one, but as it’s applied by Magit, it wraps git the way an elaborate completion script would in a shell, and it produces a kind of completion behavior that understands dependent arguments.
The completion features in all three of the most popular shells (bash, zsh, and fish) are comparable to those in TOPS-20. The difference is they do this with completion scripts that are independent of the actual command and have to be installed separately.
As I mentioned, some commands generate their own completion scripts, e.g. source <(mycommand completion bash) would install mycommand's completions into a bash session, and some argument parsing libraries include that functionality (e.g., Swift ArgumentParser). So in that sense completion definition is integrated into the command, but it's pre-baked, not as interactive as the TOPS-20 method.
Magit doesn't really just wrap git commands; it's a complex elisp application that uses git commands in its implementation. The transient elisp library is a way of giving sophisticated arguments to elisp code, not building Unix command lines.
You could combine these approaches and have a TUI tool that generated a magit-ish UI for the completion script. Macintosh Programmers Workshop had a thing called commando that read argument definitions from a resource in the tool binary and would automatically build a GUI dialog with checkboxes, dropdowns, etc.
I gather that some of the interactive usability features in csh/tcsh were copied from TOPS 10 or 20, which is why tcsh was the popular unix shell through most of the 1980s – until the rise of bash in the 1990s. previously
Not quite as cool, but Abseil's flags module exposes an --helpxml flag that spits out structured data you can parse: https://abseil.io/docs/python/guides/flags#special-flags
A lot of Apple’s first-party CLI tools work this way: their command-line flags are handled via the CoreFoundation user defaults later and you can set default values for them with the defaults tool (which inspects and modifies the persistent user defaults domains).
Can you list a few examples? This would be really helpful for something I'm looking into.
Most of the GUI administrative tools are Something Utility. There’s usually a somethingutil that accompanies it. For example diskutil, hdiutil, and so on.
Ah, I should have said, I was hoping for something CLI-only. This doesn't really change my current findings.
All of the somethingutil tools are CLI only.
I mean a command line interface to something that lacks a corresponding graphical interface.
I’m not sure why that would be a useful requirement, the existence of the GUI tool doesn’t impact the design of the CLI tool. I can’t think of many CLI tools that lack a GUI wrapper or equivalent these days.
It's not guessing what you want, it's showing you what's allowed given your input so far.
Yeah, I think this is a false dichotomy..I want to understand how it works but I still want it to be easy to use and not give me completions that won't work.
ZSH's completion system does this automatically for not repeating options (see the _arguments function and the explanation of *optspec), and it has a mechanism to define mutually exclusive arguments (see the paragraph starting with "Each of the forms above").
But I also think that encoding these very specific behaviors in a completion script is a lot of work tied to the definition of the CLI interface with questionable benefit. If the CLI interface changes regularly, it's going to be a lot of work to keep the actual behavior in line with the completion behavior.
The zsh kubectl completion goes the other way, and just calls kubectl __complete with the provided arguments to get the completions. This keeps the implementation in sync with the completion, but it's very slow (at least for me).
There is an edge case where repetition of the same option with different values is a valid way to pass a list of values.
Edit: that's actually about dependencies of options, for which it is a good suggestion.
Fish completion scripts use this. Both at the shell level (don’t suggest an option that can only appear once again) and at the completion scripts use logic level (only suggest option X if option Y has been entered or only if option Z hasn’t).
Fun sidebar: the git completion script is probably our most complex.
I’ve always found things trying to autocomplete git options to be annoying- it makes tab completion obnoxiously slow when you’re simply trying to tab complete a directory name, but also I’ve never found the need to tab complete branch names but maybe that’s a byproduct of how slow it is?
It's very easy to specify completions whose behavior differs depending on what subcommands/options you've entered so far in https://github.com/adaszko/complgen
I have a library that makes contextual, self-completion easy to manage and test: https://github.com/coxley/complete
I’ve used it for:
Like the author, I wish it was more common. It’s such a subtlety, but makes a big difference for production tooling you use a lot at work.
I wrote a Javascript package that uses parser combinators to parse command-line input. It also supports interacting with the web page during parsing, e.g. so items referenced in the command can be highlighted as it is typed.