How does the background select really work?

[j4james]

From the results we've got so far in the margin_clipping and raster_dimensions tests, I know there are some aspects of the background select that I've got wrong, but I'm still not completely sure what the correct behavior is. So before I make any more adjustments to those tests, I'll like to try and get a better understanding of the edge cases.

To start with, it looks to me like the VT340 only starts filling the background once you've output at least one pixel. So my first question is, what happens if the sequence didn't include any sixels at all? Would it just have no effect and not output anything, i.e. in these two tests, would you just get a screen that is filled with E's?

printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq\033\\n'
printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq"1;1;400;240\033\\n'

And if that's the case, what does is take for the background select to kick in? Is a 'zero' sixel enough (i.e. ?), or would you need at least one non-zero sixel (e.g. @)? Do both of these tests fill the screen with black?

printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq?\033\\n'
printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq#0@\033\\n'

And testing with an explicit raster size, do both of these fill the top left quarter in black?

printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq"1;1;400;240?\033\\n'
printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq"1;1;400;240#0@\033\\n'

Another thing implied by the current results is that a graphic new line wouldn't be enough to trigger the start of the background fill. If that's correct, then I'd expect only the bottom half of the screen to be filled with black in these next two cases. Is that correct?

printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033P2q--------?\033\\n'
printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033P2q--------#0@\033\\n'

And testing with an explicit raster size, I'd expect these two to fill the middle left portion of the screen (rows 6 to 17, columns 0 to 39).

printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq"5;1;400;240----?\033\\n'
printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq"5;1;400;240----#0@\033\\n'

Another area I'm unsure of is the handling of the zero and omitted parameters in the raster size definition. Contrary to what the DEC STD-070 documentation says, my understanding is that both zero and omitted parameters are treated as representing the maximum extent (rather than 1). If that is correct, then the two examples below should fill the top half of the screen (given an explicit height, but default width).

printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq"1;1;;240?\033\\n'
printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq"1;1;0;240?\033\\n'

And these two should fill the left half of the screen (given an explicit width, but default height).

printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq"1;1;400?\033\\n'
printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq"1;1;400;0?\033\\n'

I'm not expecting you to screenshot all of these - I'm just looking for confirmation whether they are or aren't filling the screen in the way I expect. Once I think I know what's going on, I'll try and update my main test cases to take these results into account.

[j4james]

It just occurred to me that you can't exactly cut and paste these examples into the VT340. I'm assuming it'll be easier to test with a simple shell script that runs them one by one.

#!/bin/bash

printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq\033\\\n'
printf ' 1. Screen filled with Es? '
read -sn 1
printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq"1;1;400;240\033\\\n'
printf ' 2. Screen filled with Es? '
read -sn 1

printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq?\033\\\n'
printf ' 3. Screen completely black? '
read -sn 1
printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq#0@\033\\\n'
printf ' 4. Screen completely black? '
read -sn 1

printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq"1;1;400;240?\033\\\n'
printf ' 5. Top left quarter black? '
read -sn 1
printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq"1;1;400;240#0@\033\\\n'
printf ' 6. Top left quarter black? '
read -sn 1

printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033P2q--------?\033\\\n'
printf ' 7. Bottom half black? '
read -sn 1
printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033P2q--------#0@\033\\\n'
printf ' 8. Bottom half black? '
read -sn 1

printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq"5;1;400;240----?\033\\\n'
printf ' 9. Middle left black? '
read -sn 1
printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq"5;1;400;240----#0@\033\\\n'
printf ' 10. Middle left black? '
read -sn 1

printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq"1;1;;240?\033\\\n'
printf ' 11. Top half black? '
read -sn 1
printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq"1;1;0;240?\033\\\n'
printf ' 12. Top half black? '
read -sn 1

printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq"1;1;400?\033\\\n'
printf ' 13. Left half black? '
read -sn 1
printf '\033[H\033[?7h';yes E|tr -d '\n'|head -c 1920;printf '\033[H\033Pq"1;1;400;0?\033\\\n'
printf ' 14. Left half black? '
read -sn 1

[hackerb9]

The github website gave me a textarea form I could fill in that I thought was for the response, but on the VT340 I couldn't tell I was actually editing your comment! Hah! ☺ Anyhow, the answer is "No" to the first two and "Yes" to all the rest.

[j4james]

Thanks for testing that. That makes sense. I guess it waits for either the first sixel to be ouput, or the end of the image.

[j4james]

Although now that I think about it, that doesn't explain the second test. If it was going to fill anything, I would have expected it to be limited to the 400x240 size, i.e. a quarter of the screen. So that is a bit strange.

[hackerb9]

I put a sleep in the first test and it seems to clear the screen when the first byte after q is received. (I didn't send the backslash for five seconds). It's possible there is some byte which it would not clear the screen for, but I'm not sure what.

[hackerb9]

Oh, duh. Of course, it's waiting for DECGRA. So a double quote after the q does not clear the screen.

[hackerb9]

For the second test, it clears the screen unless there is some sixel data. For example, this clears the top left corner (as soon as it receives the escape character):

printf '\e[H\ePq"1;1;400;240#0'; sleep 5; printf '\e\\'

This clears the upperleft corner when it receives the letter 'a':

printf '\e[H\ePq"1;1;400;240#0a'; sleep 5; printf '\e\\'

[hackerb9]

I think clearing the whole screen when there is no sixel data is a bug as the following should be equivalent, but correctly clears just the top left corner:

printf '\e[H\ePq"1;1;400;240"1;1;400;240'; sleep 5; printf '\e\\'

[UPDATE: new test cases show that the second DECGRA is ignored unless there is sixel data]

This clears a 400x240 square:

printf '\e[H\ePq"1;1;400;240"1;1;200;120'; sleep 5; printf '\e\\'

This clears a 200x120 square:

printf '\e[H\ePq"1;1;400;240"1;1;200;120'"1;1;100;60'; sleep 5; printf '\e\\'

This clears a 100x60 square:

printf '\e[H\ePq"1;1;400;240"1;1;200;120'"1;1;100;60#'; sleep 5; printf '\e\\'

I'm not sure what makes it sometimes work . A single hashmark (with no color number) works, which rules out my theory that the entire screen was clearing as a sign that the sixel data was malformed. Semicolons and numbers don't, so perhaps it is simply any token that takes the VT340's finite state machine out of the initial "getting sixel metadata" state.

[j4james]

This is getting really interesting. It didn't even occur to me until now that you could set DECGRA more than once.

Semicolons and numbers don't, so perhaps it is simply any token that takes the VT340's finite state machine out of the initial "getting sixel metadata" state.

Yeah, that seems like a reasonable theory.

[hackerb9]

Speaking of weird finite state machine things, maybe I'm just tired, but I don't understand how my VT340 seems to know to get out of sixel mode when I get back to the prompt when I don't send the String Terminator.

# Gives me a prompt on the VT340, xterm; gets stuck on mlterm
printf '\e[H\ePq"1;1;400;240' 

[time passes...]

Okay, I just put the VT340 into "Show Control Characters" mode (instead of "Interpret") and I can see that my shell (bash) is sending Esc [ ? 20004 h before the prompt I had set (PS1="$ "). That's the special "bracketed paste mode" invented by xterm. Apparently, both xterm and a true VT340 abort out of sixel mode (perhaps discarding graphics not drawn, yet?) when they receive the CSI (Control Sequence Introducer, Esc [).

This seems like an important quirk to test and document for VT340 sixel compatibility. It actually makes experimenting with sixels much nicer because I haven't been getting stuck in sixel mode with no clear visual indication of what's going on.

[j4james]

any token that takes the VT340's finite state machine out of the initial "getting sixel metadata" state.

I've been playing around with this on my own Sixel implementation, and the VT340 behavior seems reasonably sensible in retrospect. Sixel commands don't have a specific separator indicating where the command ends, so you generally execute a pending command when you encounter the start of the next command, or when you receive a sixel byte. Originally I would also execute any pending commands when the end of the image was reached (i.e. the ESC of the ST). But if I drop that final execute step, my behavior now exactly matches what you're seeing on the VT340.

There is a catch, though. I don't think this rule applies to DECGCI. If there's a pending palette change at the time the image is terminated, I think that would still be executed. For example, your original resetpalette script was just a bunch of DECGCI commands, and the last DECGCI wasn't followed by any other commands or sixel data. Yet that final command was still being executed (or so I assume - I think you would have noticed if the background color wasn't reset).

Apparently, both xterm and a true VT340 abort out of sixel mode (perhaps discarding graphics not drawn, yet?) when they receive the CSI (Control Sequence Introducer, Esc [).

Technically I think the ANSI/ECMA specs require that a DCS sequence be terminated by an ST, and thus shouldn't be executed if it isn't. But as far as I know, the DEC terminals treat any ESC, CAN, SUB, or C1 control as a valid terminator. This is mentioned in 3.5.4.4 of the STD-070 manual, and I know the VT240 worked that way, so I assumed the VT340 did as well. And in the case of Sixel, the image is already being output as it's received, so aborting doesn't make much sense.

That said, none of the open source terminal emulators handle progressive output (at least that I'm aware of), and many of them do actually abort the image if it's not terminated with an ST (i.e. they don't display anything at all). But that's a bug as far as I'm concerned. There's going to be software that just won't work, because it's dependent on the behavior of the original DEC terminals. I know I've encountered old Sixel images and DECDLD fonts that aren't "correctly" terminated.

This seems like an important quirk to test and document for VT340 sixel compatibility.

Yeah, it's definitely worth testing, but I didn't think there was much doubt about the behavior, and initially I've been concentrating on testing the edge cases for which there wasn't any clear documentation. Then again, I've frequently been wrong about VT340 behavior which I thought I understood perfectly.

[hackerb9]

Technically I think the ANSI/ECMA specs require that a DCS sequence be terminated by an ST, and thus shouldn't be executed if it isn't. But as far as I know, the DEC terminals treat any ESC, CAN, SUB, or C1 control as a valid terminator. This is mentioned in 3.5.4.4 of the STD-070 manual, and I know the VT240 worked that way, so I assumed the VT340 did as well. And in the case of Sixel, the image is already being output as it's received, so aborting doesn't make much sense.

I hadn't known that those characters could stand in for ST. Yet another good test to add to the list for any future sixel implementations.

[j4james]

FYI, I've included a test for different string terminators in PR #8. That test also checks what happens when you embed other control characters and unexpected command/parameter characters in the control string.

[hackerb9]

Results are in unexpected.png. Note that the backwards question mark is the VT340 way of indicating an unknown character.