Can garbage to stdout cause the console's input mode to change?

[vefatica]

TCC 30.00.22 x64
Microsoft Windows 10 Pro for Workstations
10.0.19045.3086 (2009, 22H2)
WindowsTerminal 1.18.2305.26002 (preview)

Using Cygwin's CAT.EXE ((GNU coreutils) 9.0), if I CAT some binary file (by mistake the first time) the console's input mode is very likely to be changed to include ENABLE_VIRTUAL_TERMINAL_INPUT. Whether or not that happens depends on the binary file and is consistent for a given file. This has a devastating effect on my shell (TCC) but it also happens in PowerShell/PWSH (apparently without the devastating effect). For example, in PowerShell, before and after issuing
d:\cygwin\bin\cat.exe d:\wt\exe\WindowsTerminal.exe ... before ...

V:\> conmode /i

Console input mode flags (0x1F7)

    ENABLE_PROCESSED_INPUT (0x1)
    ENABLE_LINE_INPUT (0x2)
    ENABLE_ECHO_INPUT (0x4)
    ENABLE_MOUSE_INPUT (0x10)
    ENABLE_INSERT_MODE (0x20)
    ENABLE_QUICK_EDIT_MODE (0x40)
    ENABLE_EXTENDED_FLAGS (0x80)
    ENABLE_AUTO_POSITION (0x100)

after ...

V:\> conmode /i

Console input mode flags (0x2E8)

    ENABLE_WINDOW_INPUT (0x8)
    ENABLE_INSERT_MODE (0x20)
    ENABLE_QUICK_EDIT_MODE (0x40)
    ENABLE_EXTENDED_FLAGS (0x80)
    ENABLE_VIRTUAL_TERMINAL_INPUT (0x200)

The same happens in a Windows console. It does not happen if I use another means of sending the file to stdout ... TCC's TYPE, PowerShell's TYPE, an older (32-bit) Gnu CAT.EXE ((GNU coreutils) 5.3.0), WSL CAT (each of those producing visually different output).

I wouldn't expect Cygwin's CAT.EXE to do that on purpose. And whether or not it happens depends on the binary file. So I must ask whether sending garbage to stdout could cause it.

[DHowett]

Excellent question! The trick with anything running under Cygwin (and MSYS2 by extension) is that it is not dumping text to stdout. One of its default modes of operation simulates a terminal using the Win32 console APIs by processing stdout for control sequences.

And there are absolutely control sequences that change how input is generated.

It's not a stretch to imagine that Cygwin is detecting one of those control sequences and translating it into a request for SetConsoleMode. The bug is moreso that it is not restoring the right modes when it exits.

[vefatica]

OK, thanks! Do I have any choice regarding Cygwin's default MO?

I only got Cygwin to get 64-bit versions of my old GnuWin32 tools. To be honest, in normal use, they don't make any difference. And CAT is one I almost never use.

[DHowett]

Hmmm. I'd be interested in seeing what $TERM is set to inside Cygwin. As of version 3.1 or 3.2, they added support for the Windows 10 console. If that support is being used, TERM will be xterm-256color. If not (and they are using translation,) TERM will be cygwin.

[vefatica]

If I just start d:\cygwin\bin\cat.exe (it'll wait to process stdin), TERM is not in it's Windows environment. If I set TERM in my shell before starting CAT.EXE, it's inherited, but using either of the values you mentioned doesn't alter the behavior in question. [Was that test meaningful?]

[ClaireCJS]

This bug has plagued me for years, but I keep stubbornly using cat.exe instead of the internal type command anyway.

The end-result is, tab filename completion no longer works, and the shell has to be terminated and re-opened entirely. It's awful.

[vefatica]

OT: What is ENABLE_AUTO_POSITION (0x100)? I see it in OpenConsole and not in Conhost. What does it do?

[zadjii-msft]

Interesting. It's not really used for anything in the current codebase.

https://github.com/search?q=repo%3Amicrosoft%2Fterminal%20ENABLE_AUTO_POSITION&type=code

This would require much more historical digging to figure out what it was.

[DHowett]

not really used for anything

It's translated into CONSOLE_AUTO_POSITION, which is used.

Promoted:

terminal/src/host/getset.cpp

Line 56 in 2f0d3dc

WI_SetFlagIf(mode, ENABLE_AUTO_POSITION, WI_IsFlagSet(gci.Flags, CONSOLE_AUTO_POSITION));

Used:

terminal/src/interactivity/win32/menu.cpp

Line 323 in 2f0d3dc

pStateInfo->AutoPosition = !!(gci.Flags & CONSOLE_AUTO_POSITION);

terminal/src/interactivity/win32/window.cpp

Lines 267 to 301 in 2f0d3dc

	if (!WI_IsFlagSet(gci.Flags, CONSOLE_AUTO_POSITION))
	{
	//if launched from a shortcut, ensure window is visible on screen
	if (pSettings->IsStartupTitleIsLinkNameSet())
	{
	// if window would be fully OFFscreen, change position so it is ON screen.
	// This doesn't change the actual coordinates
	// stored in the link, just the starting position
	// of the window.
	// When the user reconnects the other monitor, the
	// window will be where he left it. Great for take
	// home laptop scenario.
	if (!MonitorFromRect(rectProposed.as_win32_rect(), MONITOR_DEFAULTTONULL))
	{
	//Monitor we'll move to
	auto hMon = MonitorFromRect(rectProposed.as_win32_rect(), MONITOR_DEFAULTTONEAREST);
	MONITORINFO mi = { 0 };
	//get origin of monitor's workarea
	mi.cbSize = sizeof(MONITORINFO);
	GetMonitorInfo(hMon, &mi);
	//Adjust right and bottom to new positions, relative to monitor workarea's origin
	//Need to do this before adjusting left/top so RECT_* calculations are correct
	rectProposed.right = mi.rcWork.left + rectProposed.width();
	rectProposed.bottom = mi.rcWork.top + rectProposed.height();
	// Move origin to top left of nearest
	// monitor's WORKAREA (accounting for taskbar
	// and any app toolbars)
	rectProposed.left = mi.rcWork.left;
	rectProposed.top = mi.rcWork.top;
	}
	}
	}

[zadjii-msft]

ENABLE_AUTO_POSITION

CONSOLE_AUTO_POSITION

ngl I didn't even see that those were different. I need more sleep 😪