~/ completes to \~/ on zsh

[FlorianPommerening]

The alias ~ for the home directory is escaped on zsh.

To reproduce

Test environment:

• Ubuntu 24.04
• Python 3.12.3
• argcomplete 3.1.4-1ubuntu0.1 (this is 3.1.4 with a patch back-ported from 3.3.0 to make it compatible with Python 12.3)
• zsh 5.9

Content of foo.py

#! /usr/bin/env python3
# PYTHON_ARGCOMPLETE_OK

import argcomplete, argparse

parser = argparse.ArgumentParser()
arg = parser.add_argument("file")
arg.completer = argcomplete.completers.FilesCompleter()

argcomplete.autocomplete(parser)

./foo.py ~/.bashr<TAB>

Expected completion is ./foo.py ~/.bashrc and this is what happens on bash. However on zsh, I get ./foo.py \~/.bashrc.

In both cases, the FilesCompleter returns the path as ~/.bashrc (not escaped), so the escaping happens later on in the pipeline. In particular, this is part of the zsh built-in _describe which in turn calls compadd to add the string ~/.bashrc to the list of suggestions. At this point the string is escaped. There are options to prevent this escaping that can also be passed to _describe but it is not clear to me, which one to use (https://zsh.sourceforge.io/Doc/zsh_us.pdf, chapter 19.3 for compadd and 20.6 for _describe). For example, -Q would disable escaping completely but this would be an issue for paths with spaces. Then something like -f -W \~ might work but it assumes that the returned values are paths.

[chrisdiamand]

Hi, I'm experiencing this too - the consequence on zsh being that whatever command is being called fails, because the filename it's been given (with the escaped ~) doesn't exist! I have to go back and remove the extra \ from every filename in the command, which really negates the time saving of completion.

I've done some quick tests and this happens both when using argparse.FileType and when using type=string, either with or without setting completer to FilesCompleter.

[kislyuk]

Thanks for your input. I can reproduce the issue but it appears cosmetic - while unnecessary, the escaping is semantically correct, right? Do you see any errors or completion blocking issues arise out of this?

I agree that the escaping is unfortunate and would be glad to get rid of it, but I'm not sure how to do that yet. As @FlorianPommerening points out, zsh makes a complicated set of assumptions when escaping that I'm not fully confident we can control.

[chrisdiamand]

Hi, thanks for looking into this!

Unfortunately it's not just a cosmetic thing - the escaped version has a different meaning than the non-escaped version. Specifically, zsh won't expand the ~ when it's escaped.

For example, in the example above, ./foo.py ~/.bashrc would be expanded to ./foo.py /home/whatever/.bashrc before foo.py is run. But the escaped version (./foo.py \~/.bashrc) is left as-is, so foo.py is passed the literal ~ character, which most programs don't expand themselves.

This means it would try to open a file in a subdirectory of the working directory called ~, rather than having ~ expanded to $HOME.

[kislyuk]

Ah, you're right. OK, into the mines of zsh completion I go.

[chrisdiamand]

What a way to spend a Sunday... thanks and good luck!

[kislyuk]

I'm thinking this is part of a broader issue where we use a bash subprocess to generate filename and directory completions instead of communicating back to the shellcode that we should ask the shell to do that directly. This will take a little bit of rearchitecting.