Embrace Standardized Color Control

[mitsuhiko]

I was trying to get people to embrace this crate before as a unified way to control colors but it did not get far: https://crates.io/crates/clicolors-control (that crate also turns on ansi colors for the windows console as an added bonus)

The idea was that you have one common crate that turns on and off colors for the entire cli app.

I'm fine finding a different mechanism for this but right now controlling colored output is really not a particularly fun experience.

Anything got some thoughts on this matter?

[killercup]

Relevant internals thread: https://internals.rust-lang.org/t/terminal-platform-abstraction/6746

[BurntSushi]

I've looked at clicolors-control before, but didn't use it for a variety of reasons on which reasonable people may disagree:

• It doesn't look at the TERM environment variable. I personally don't really care about the clicolors standard, so I think it's fine to use it, but it shouldn't be used to the exclusion of other things. When people set TERM=dumb, for example, then people expect that to be respected. (I've even found that some environments that don't support colors will do this automatically, and if you don't respect it, you end up emitting a bunch of escape codes.)
• Its tty detection on Windows is insufficient for my needs. I need it to work in MSYS2 terminals. The atty crate does this using a hack.
• My termcolor crate (which predates clicolors-control, AFAIK) already did everything that clicolors-control does, and assimilates it with common end users inputs via the ColorChoice type. (With one exception: only recently did it acquire the ability to enable ANSI escapes on Windows 10.)

I would in principle support coalescing around one crate that just did color detection, but on that same token, I'd also like to see people coalesce around one crate for dealing with a Windows console for coloring, e.g., wincolor.

[mitsuhiko]

@BurntSushi for what it's worth I do not really want to defend either the crate nor the defaults. I mostly just want a crate that holds the state of color on/off with a sensible default.

My termcolor crate (which predates clicolors-control, AFAIK) already did everything that clicolors-control does

The reason I never ended up using termcolor was that it does not actually (to the best of my knowledge) store the state of color on/off. It just has the auto color choice which derives the default for the writers. You cannot query it and ask: does the user want colors.

(My goal effectively is that I can parse --color=never and then tell the entire ecosystem of crates I use to please not emit colors)

[richard-uk1]

The way I would like to see terminal colours work is a wrapper around std(out/err) that interperets ansi escape sequences. If we are on windows before windows 10, we have to use the win32 api to process escape sequences, but on most platforms we can just pass them through. There is also terminfo/cap that should be respected.

If we do this, everyone just uses ansi escape sequences, and the wrapper can choose to discard them based on anything (TERM, configuration, etc.).

Idea from this post.

(My goal effectively is that I can parse --color=never and then tell the entire ecosystem of crates I use to please not emit colors)

Using this option, disabling colors would happen in the consuming crate, all crates further down the chain just write their ansi escape sequences as usual.

A bit of code:

struct AnsiPolyfillWrapper<W: io::Write>(W);

impl<W: io::Write> io::Write for AnsiPolyfillWrapper<W> {
    fn write(&mut self, buf: &[u8]) -> Result<usize> {
        // read buf and if necessary modify the escape codes, with many write calls, otherwise
        self.0.write(buf)
    }
}

[BurntSushi]

There is also terminfo/cap that should be respected.

I explicitly chose to not respect this, at least for things like colors or other simplistic styling like underlining or bold. The reason why is because when I did use a library that respected terminfo, I got numerous bug reports where the wrong escape codes were being emitted. Instead, I chose to follow in the footsteps of GNU grep: just stick to the basics. GNU grep is widely deployed, and I never hear anyone complaining about whether its colors work or not, which is compelling evidence from my perspective that it has the right approach here.

For CLI tools that need to do more extensive manipulation of the terminal, then this may be the wrong calculus.

In general, I agree that an ANSI interpreter would be great, and it would, for example, slide nicely into termcolor's existing API (while simultaneously removing the need for the buffer and buffer writer types). It's a nice-to-have though IMO, and we can make progress without it. :-) In particular, the kind of support you're talking about would require a dedicated maintainer.

[richard-uk1]

@BurntSushi what do you think are the most pressing issues in this area? Sorry if this question is answered elsewhere.

[mitsuhiko]

@derekdreery you still need to know if colors are on or off for other reasons. For instance I shell out to other processes which will typically not detect color correctly so I want to propagate my understanding of if color is on or off to that subprocess.

WRT to the wrapping stream: I agree that would be nice. Sadly right now there is only an undocumented API used by rusttest to intercept prints.

(Also now that windows 10 supports ansi colors after enabling i'm hoping we can just pretend ansi color is a thing and get rid of more complex wrappers around it)

[BurntSushi]

@derekdreery The hardest possible problem: getting people to agree on one library. It may be the case that you need an ANSI interpreter to do that, because termcolor's API today is not nearly as simple as crates that only handle ANSI formatting, and that annoys people enough to go out and write their own library.

@mitsuhiko's idea about at least getting all of them to agree on whether colors are enabled or not (and permit the caller to control) is a really nice middle ground and possibly much easier to achieve. I would certainly be on board with doing that for termcolor (subject to my concerns listed above).

[Screwtapello]

The idea of interpreting ANSI codes alarms me for a number of reasons:

• The number of potential ANSI codes somebody might conceivably want to send is very large, and drawing any hard line will rule out some number of reasonble use-cases
• What happens with things that look like ANSI codes but this crate doesn't know how to handle?
  • If they're passed through, you're likely to get output that works perfectly on some terminals and blurts ANSI goo in others
  • If they're silently ignored, that makes formatting problems difficult to debug
  • The library could raise an error, but is it worth killing the entire operation over an output formatting failure? Even if the culprit is just improperly-escaped user-input?
• "half assed ANSI code handling" has caused problems before (although in that case the Rust program was the producer of such codes, not the consumer)
• Even with helper functions with friendly names like make_green_text(), if your library's officially-supported API includes "put raw ANSI codes in your string literals", then people will do that, and raw ANSI codes are not an API surface that existing API tools like rustdoc are equipped to handle gracefully
• The reason that raw ANSI codes are a popular API is because it's so easy to bang strings together, and so much more difficult to work with a more structured API. This is also the reason for the popularity of cross-site-scripting attacks and SQL injection vulnerabilities. ANSI codes probably aren't as security sensitive as those other things, but still worth considering.

All that said, it may still be the case that the ease and flexibility of a custom ANSI parser outweighs all the downsides, in which case fair enough.

However, taking a cue from the "cross site scripting" analogy, I'd be interested to see/use a terminal colour API based on the ideas of HTML templating libraries like Elm's HTML DSL, ScalaTags or rust-tags. That is, building up a type-safe data-structure that represents text and its formatting, which can be stored and later serialized to ANSI-codes or Windows console API calls or HTML with inline CSS or whatever. That would hopefully have all the thread-safety advantages of ripgrep, with a more ergonomic and teachable API.

[BurntSushi]

@Screwtapello you raise a lot of good points I hadn't considered! Briefly, I will just respond to one thing here, and that's the idea of building up something more structured. In particular, for tools like ripgrep, printing output is a performance sensitive aspect of execution. It is likely that any overhead associated with emitting colors would need to be avoided, and I imagine execution templates probably wouldn't be acceptable. With that said, a convenient API doesn't need to solve every use case, so long as there are lower level APIs for when that is necessary.

[mitsuhiko]

The wider discussions a bout ansi, terminal abstractions etc. are quite complex which is why i thought about reducing this problem to initial color detection and control.

I think it might make sense to have a crate like clicolors control that just has a flippable global and then maybe various compile features to pick a sensible default (on, off, CLICOLOR, term detection etc.)

Then it should be easy for everybody to use it as a base and the rest of the discussion can be held separately.

[richard-uk1]

@Screwtapello Thanks for explaining all that! I had not considered ideas like cross-site scripting.

What I'm trying to define is the most general, minimal level, and fast API that can do everything that's required.

The number of potential ANSI codes somebody might conceivably want to send is very large, and drawing any hard line will rule out some number of reasonble use-cases.

We would draw a line somewhere, probably text styling and cursor movement to start with. This line could be moved in future, and if people used the library it would show where the gaps are and what features are most missed. These could be prioritised.

What happens with things that look like ANSI codes but this crate doesn't know how to handle?

I think there are a few options here. I believe that in production (release mode) the correct thing to do is to strip out these sequences, but in debug mode it may be useful to log their presence.

If they're silently ignored, that makes formatting problems difficult to debug

If someone comes to you with a formatting issue, you can hand them a version of the program compiled to write out a warning when it comes across an unexpected sequence.

"half assed ANSI code handling" has caused problems before (although in that case the Rust program was the producer of such codes, not the consumer)

I'd argue this would just be a bug that should be fixed. Any implementation may have bugs.

Even with helper functions with friendly names like make_green_text(), if your library's officially-supported API includes "put raw ANSI codes in your string literals", then people will do that, and raw ANSI codes are not an API surface that existing API tools like rustdoc are equipped to handle gracefully

I agree. This library should not be for user consumption. An alternative library should be advertised that implements a programmatic API, using the type system to prevent invalid input. But we still need the implementation of that API, and I propose that this is the best way to do it, or at least that it is worth investigation.

This is also the reason for the popularity of cross-site-scripting attacks and SQL injection vulnerabilities.

I need to think more about this to fully understand the risks. I think it would be the role of the higher-level API to sanitize any input, so that escape sequences are only emitted from the API calls. This would need careful auditing.

So, in summary you would have 2 levels

1. (lower) escape sequences polyfill

• fast (this is a guess because I imagine you could write a cache-friendly wrapper around the write calls, would need to check if its true or not)
• flexible (can be sent over stream before being converted into platform-specific, pre-serialized)
• complicated, and error prone if you write your own escape sequences

2. (higher) programmatic API
   • safe (type system and sanitization)
   • simpler, more ergonomic
   • slower (maybe, or will api calls just get inlined into string operations)
   • risky (sanitation needs to work, is there a difinitive list of escape sequences, or patterns of escape sequences that need to be stripped).

If you went straight for a higher level API you still have the issue of what to do with escape sequences in the input. Should you remove them, or leave them in? If you remove them, how do you know you are getting them all?

@mitsuhiko what are the specific things that are missing now that you would like to see as a minimum?

[mitsuhiko]

@derekdreery there needs to be an agreed upon way to turn color on and off. clicolors-control exists but nobody uses it and @BurntSushi outlined some of his reasons for not doing it. So what I can propose right now is to take that crate, incorporate his suggestions but I won't be able to force the rest of the community to use it :)

[richard-uk1]

Could you simplify it to:

• If CLICOLOR == '0' force disable color
• If CLICOLOR != '0' force enable color
• If CLICOLOR is unset, make a guess (using TERM etc.)

[iquiw]

FYI: There is an attempt to make standard with NO_COLOR environment variable.
http://no-color.org/

[richard-uk1]

I've played with this a bit, reading the xterm guide. I've also been reading the standard. My plan is to implement the ansi standard, at least to recognise all escape sequences specified there. Then I can process a subset of them and skip the rest.

For the issue with hostile escape sequence injection, any utf-8 sanitized text will not contain escape sequences. Therefore it is sufficient to check that input is utf-8 (see the xterm guide near the beginning for more details).

[BurntSushi]

@derekdreery If I'm understanding you right, I'm not sure that will work. In particular, assuming or requiring that output (or input) is UTF-8 is unfortunately inappropriate in almost any UNIX command line tool.

[richard-uk1]

@BurntSushi my argument is that you can avoid injection by using utf8, but that if you are not then there may be escape sequences present. My approach (at least to start with) will be to just strip any escape sequence as defined in the spec (including things like STX), and add back in the ones that I'm willing to handle for the given platform.

I think what I'm doing is an experiment at this point. I think having a working (by my probably incorrect definition) library will help to spark more debate.

[Screwtapello]

For the issue with hostile escape sequence injection, any utf-8 sanitized text will not contain escape sequences.

I think you're talking about C1 control codes, which overlap with UTF-8 extension bytes. While it's true that a C1 control code like 0x9B (CSI, Control Sequence Introducer) is not directly valid in UTF-8, terminals date back to a time when only 7-bit ASCII was reliably available, so every C1 control-code has a 7-bit encoding of ESC followed by an ASCII character. Thus, any ANSI code defined to begin with CSI can also begin with ESC [ (0x1b 0x5b), and that is valid UTF-8.

[richard-uk1]

@Screwtapello you are correct, I'm still learning :).

[mitsuhiko]

@iquiw there was an earlier attempt to standardize on CLICOLOR which managed to gain some traction and that's what clicolors-control currently uses: https://bixense.com/clicolors/

[vitiral]

One thing I would like to point out when developing a terminal color API is the annoyance of testing it. It should be easy to:

• Represent the formatting as data, i.e. as a list of structs which may have embedded structs
• It should be possible to serialize/deserialize these structs using serde so that tests can useyaml or some other format for writing expected output.
• Ideally there would even be some kind of "html" like language for representing the formatting.
• It should be possible to mutate the formatting, especially "force plain". It is really annoying to have if/else blocks depending whether the user specified --plain. I want to just be able to call text_items.map(|t| t.set_plain()) and be done with it.

I wrote termstyle to address some of these issues, but I thought it would be good to bring them up here. (not saying termstyle is the be-all-end-all -- I've only just started using it myself).

I totally agree that the actual coloring should be done through a single low-level (standardized) crate. I didn't actually know the trick with windows and I'm now inclined to use ANSI escape codes for all platforms.

[uazu]

For using colours in CLI output, also consider that for example less has a pass-through mode for only ESC [ ... m sequences using the -R option (by default). This works solidly, e.g. with line-wrapping. So if you want to be compatible with less (as git and grep and so on are), then you limit yourself to ESC [ ... m. So this is relatively simple to handle, e.g. embedded in strings, or for stripping out or skipping when counting visible characters.

Terminfo may include other control codes in its colour sequences, which are fine for a TUI but not what you want in this case.

[uazu]

Considering that a CLI tool (but not a TUI) generally needs to handle having its output redirected to a file and then read later in a pager, this really cuts down on what ANSI sequences need to be handled.

Other ANSI sequences might be used inside a readline-like library, but this is not visible outside of that library, and wouldn't get into a log file from non-interactive use of the CLI tool (e.g. where there is no use of stdin, or stdin is redirected from a file).

[richard-uk1]

If anyone's interested on progress I'm currently gaining the ability to recognise all ansi escape sequences according to the spec. At that point I'll be able to strip escape sequences. After that, I will start to interperest some of the more simple sequences like colour.

…

On 9 Mar 2018 18:20, "Jim Peters" ***@***.***> wrote: Considering that a CLI tool (but not a TUI) generally needs to handle having its output redirected to a file and then read later in a pager, this really cuts down on what ANSI sequences need to be handled. Other ANSI sequences might be used inside a readline-like library, but this is not visible outside of that library, and wouldn't get into a log file from non-interactive use of the CLI tool (e.g. where there is no use of stdin, or stdin is redirected from a file). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/rust-lang-nursery/cli-wg/issues/15#issuecomment-371900673>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABU-XqoPo2VRS00EHFazr7Dtj91uxyFtks5tcsfogaJpZM4SZwS0> .

[luser]

For the sorts of tools I wrote, declaring that you can only have color support on Windows 10 is just fine. Most command-line tools tend to be developer-focused, and most developers aren't running old versions of Windows anyway. I think as long as tools work on older versions of Windows, color is simply an added bonus, so I don't think we should be writing foundational crates that jump through hoops to support color in that situation.

If anyone's interested on progress I'm currently gaining the ability to
recognise all ansi escape sequences according to the spec. At that point
I'll be able to strip escape sequences. After that, I will start to
interperest some of the more simple sequences like colour.

You might be interested in the strip-ansi-escapes crate I put together recently for sccache. sccache runs compilers and captures their stdout/stderr and then outputs it elsewhere, so the standard "is stdout a tty" checks don't work well. With this crate I can force color output on from compilers and then do the tty check when it's time to output and strip escapes at that point. (My crate uses the vte crate for the actual parsing, so I have fair confidence that it does the right thing.)

[BurntSushi]

Most command-line tools tend to be developer-focused, and most developers aren't running old versions of Windows anyway.

Do you have numbers on this?

[BurntSushi]

and the fact that supporting colored output on Windows versions prior to Windows 10 is a significant amount of extra work, I don't think it's a worthwhile investment.

It's already done. But I don't know what we're arguing about. You aren't going to convince me to stop supporting <Windows 10 users.

[lzybkr]

Here is some data - the numbers come from what I would call early adopters using the open source version of PowerShell. I'm not sure what the numbers are exactly - I think they are process launch, not number of users, but you can see 4% (80K) on pre-Win10 systems.

[softprops]

Sorry if this was already mentioned before but I want to just throw it out there. Are there inspirations that answer some of the questions outlined here that can be derived from prior established standards in other language communities? Color control standards doesn't necessarily feel like rust specific territory. If such standards exist I would opt for rust to find ways to honor them. As a cli user, I dont typically expect to find ux to vary by the language a cli was written in. In fact I prefer it didn't! If such standards do not exist in other communities that have been have mature cli ecosystems, does this WG feel this is still an important area to standardize on?

[BurntSushi]

@softprops There are various things I'm aware of like the cli colors spec, but I don't know what kind of adoption it has. For me, when I wrote termcolor, I did a very rudimentary survey of command line applications that I knew were in widespread use, like GNU grep. For the most part, that logic is captured here: https://github.com/BurntSushi/ripgrep/blob/d7c9323a689239ac8b29baba63b1c3a26e12afda/termcolor/src/lib.rs#L135-L147 --- There isn't much to it.

Separately from that, there is the terminfo database, but I ran into problems that I didn't know how to solve, and it seemed like overkill for just needing to emit colors.

I personally would be very weary of standards in this area. Actual behavior of common command line tools wins the day IMO.

(There are other questions, which clicolors-control aims to solve, around the notion of the main program being able to control whether its dependencies emit colors in their output. It's an acknowledgment of the idea that we will need to live with multiple coloring libraries, but maybe it will be OK if they all respected the same global state, not unlike how the log crate works.)

[kbknapp]

but maybe it will be OK if they all respected the same global state, not unlike how the log crate works.

I personally like this idea as ultimately I think it's the end user who needs to decide if they want color or not (and when). The only issue I see with a shared global state such as clicolors-control is one of complexity and potential for the chain to be incorrect somewhere along the line.

I might be misunderstanding, but I see it like this:

Whereas maybe something more like log (just a trait that all libs impl) would look more like this:

And could benefit from fewer branches (maybe?) and moves the global state from clicolor-control to the lib implementing the trait.

Maybe I'm way off and need coffee, but just some thoughts 😜

[epage]

Oh, look, this was already discussed. Wish I had checked before researching this topic. When this thread started, I had skipped over it because I hadn't gotten into colored output yet for my applications and knew I was over my head. Now its being a pain for me and am looking to do something about it. styling crate authors are still interested in the global state, I'm game for implementing it.

Now onto my research:

Inspiration cases

• I want to respect all color standards in typos in a consistent way (regardless of whether I'm printing, env_logger, or clap)
• I want assert_cmd to print error info in color, particularly diffs generated by predicates but the API is stateless, having the user tell us to do color doesn't really fit in, and users having to check the right settings and configure this in every #[test] is onerous

Aspirations

A developer can quickly adopt color within their CLI that follows best practices

1. API is easy to adopt
2. User sees consistent behavior, regardless of implementation (arg parsing, logging, etc)
3. Doesn't display color when piping, unless asked
4. Will still display color on the non-piped stream
5. Respects capabilities of the terminal
   • Gracefully degrade when color mode (truecolor, 8bit, 3bit) isn't supported
   • For more on truecolor, see https://gist.github.com/XVilka/8346728
   • On Windows, either supporting the old color API or enabling ANSI support (and disabling colors when not supported)
6. User can control through expected color argument (--color[=<always|auto|never>])
7. Respects CLICOLOR, CLICOLOR_FORCE, NO_COLOR

A developer can get richly formatted test-failure messages

1. Respects conventions from above
2. Trivial to take advantage of across all tests

Other considerations: performance should not be so bad that it is noticed (like stdout locking on println)

Survey of ecosystem

Capability crates

• terminfo
  • Looks up and loads termcap database
• termbg
  • Detects type of terminal (wide range)
  • For some terminals, it directly asks
  • For others, it does a print/read cycle to see how the escape sequences are handled
• term (also under styling)
• clicolors-control
  • Detects interactive stdout (not stderr)
  • Enables / checks for ANSI support on Windows
  • Respects CLICOLOR and CLICOLOR_FORCE

Styling crates

• owocolors
  • Per-styling disable function per TTY (caller must know intended TTY)
  • ANSI only
• yansi
  • Global disable, irrespective of TTY
  • Provides function for enabling ansi colors on Windows
  • ANSI only
• termcolor
  • Auto-detect on TERM and NO_COLOR
  • Supports older Windows terminals
  • ColorChoice can be specified when building a writer
• ansi_term
  • Supports byte strings
  • Provides function for enabling ansi colors on Windows
  • ANSI only
• nu_ansi_term
  • Extension of the above
• colored
  • Respects CLICOLOR, CLICOLOR_FORCE, NO_COLOR
  • Checks stdout tty status, applies to all coloring
  • Disable colors through feature flag
  • Provides function for enabling ansi colors on Windows
    • Does not impact global color state, does not report failure so caller can change global color state
  • ANSI only
• term (also under capabilities)
  • Errors if displaying colors beyond what terminal supports
  • Supports Windows
• term-painter
  • stdout only
  • Not thread safe
  • Ignores errors from term (says comment, can't find where this happens though)
  • Supports Windows
• bunt
  • Replaces println and friends
  • Wraps `termcolor, see above
  • Supports clearing a style, to help with user-defined color palettes that degrade

Color logic

• colorparse: parse a git-style color string into ansi_term::Style
• ansi_colours for converting RGB to/from 8bit
• strip-ansi-escapes

"Middleware"

• clap2
  • color feature flag
  • Runtime color choice (always, auto, never)
  • Auto looks at TERM=dumb and tty for stderr (despite also printing to stdout)
• clap3
  • color feature flag
  • Runtime color choice (always, auto, never)
  • Auto looks at tty for stderr (despite also printing to stdout) and delegates the rest to termcolor
• structopt
  • Like clap2 except runtime color choice flag has to be passed into a proc-macro attribute on a struct/enum which is awkward and non-obvious
• eyre
  • Allows customizing theme, so user could use a blank theme
  • (built n owo_colors v1 which doesn't have if_stdout_tty / if_stderr_tty)
• trybuild
  • Auto-detect on TERM and NO_COLOR (hard codes to termcolors ColorChoice::Auto)
• env_logger
  • stdout/stderr detection (for whichever is the target)
  • Callers can customize behavior (called WriteStyle instead of ColorChoice)
  • Auto-detect on TERM and NO_COLOR

Applications

• rg
  • Auto-detect TTY, TERM, and NO_COLOR (builds on termcolor)
  • Has --color=<when> flag
  • clap2's color is turned off because it uses a different color crate
  • Wraps stdout for color choice (doesn't seem to write to stderr on its own)
  • Oddly, it looks like TTY detection is done twice, at the arg level and within the stdout wrapper
• bat
  • Turns off clap's coloring if non-interactive stdout (even if clap prints to stderr) or if NO_COLOR
  • Has --color=<when> flag
  • Failure to set ANSI support on windows does not impact color support
  • For own printing, color=auto checks for interactive stdout and NO_COLOR
  • Additionally checks for truecolor support via COLORTERM (degrades RGB to 8bit via ansi_colours)
  • For errors, unconditionally prints coloring
• hyperfine
  • Turns off clap's coloring if non-interactive stdout (even if clap prints to stderr)
  • Has --style=<type> flag which includes coloring
  • For own printing, overrides --style if stdout is not interactive
  • For errors, unconditionally prints coloring
• nu
  • At a glance, seemed to do colors unconditionally
• cargo
  • Default initializes the env_logger (interactive stdout/stderr, TERM, NO_COLOR detection)
  • Default initializes clap
    • Auto looks at TERM=dumb and tty for stderr (despite also printing to stdout)
  • --color=<when>
  • term.color in config file
  • Auto-detect on TERM and NO_COLOR, and if stderr is interactive, even if printing to stdout

Open questions

• Trade offs of terminfo vs detection through env vars?
  • burntsushi:
  • I explicitly chose to not respect this, at least for things like colors or other simplistic styling like underlining or bold. The reason why is because when I did use a library that respected terminfo, I got numerous bug reports where the wrong escape codes were being emitted. Instead, I chose to follow in the footsteps of GNU grep: just stick to the basics. GNU grep is widely deployed, and I never hear anyone complaining about whether its colors work or not, which is compelling evidence from my perspective that it has the right approach here.
  • For CLI tools that need to do more extensive manipulation of the terminal, then this may be the wrong calculus.
• What is the tipping point for recommending supporting non-ansi coloring for Windows?
  • ANSI coloring tends to make it easier to have a nice API
  • Non-ANSI means you support older Windows versions

Challenges

• Per TTY detection
• Knowing what TTY you are rendering for
• Following conventions consistently across crates which might use different styling crates which don't specify their auto-policy, requiring digging into the code
• Manual work to tunnel env vars + CLI arg to logging, error handling, clap, and through your own program
• Some color support comes through indirect dependencies (assert_cmd calling predicates calling difference)
• Some scenarios, like tests, are more hassle to setup correctly (duplciating through all tests)

Proposed solution:

Crates:

• A singleton to control color support
  • Allow style crates to read the styling choice
    • Must be breaking-change resistant so only one version shows up in dependency tree
    • Report capabilities on a TTY-by-name, stream, or "lower common denominator" basis
  • Allow main to override it, based on command line
  • Expose the implementation details as other crates to allow customization / testing but don't expose to in this crate for API stability
• Provide a clap-color crate for providing a standard way of specifying color in CLI
• Maybe explore a shared definition for colors/styling with graceful degrading
  • Ideally as much is const-fn as possible

All of this is subject to adoption, see XKCD #927

This ends up being similar to what was proposed earlier when WG-CLI discussed this

Other ideas

• Customize color detection via feature-flag system similar to https://github.com/That3Percent/firestorm (if that even works still with the new feature resolver)
• Configure logging via Existential types with external definition

[pksunkara]

You forgot console in styling crates list.

A singleton to control color support

clicolors-control crate is an attempt at it.

[epage]

You forgot console in styling crates list.

Huh, surprised I missed one that has so many downloads. I suspect it doesn't add anything new but I'll try to take a look at it earlier

A singleton to control color support

clicolors-control crate is an attempt at it.

Yes, as I mention in the beginning, I found this thread (and clicolors-control) after doing all of this and only did minor touch ups to acknowledge it. I need to spend some more time on it to see how well it meets the aspirations listed out.

[pksunkara]

A relevant comment regarding that crate's current status. console-rs/console#90 (comment)

[epage]

Why do you feel checking tty is too much?

[epage]

Things missing from clicolors-control

• Treat stdout and stderr distinctly
• NO_COLOR support
• Truecolor support

The API needs to be able to detect level of color support given either stdout/stderr (names), a stream, or "lowest common denominator".

If we build on clicolors-control, We can introduce these without a breaking change, carefully choosing what new behavior the old APIs map to, but I suspect we should consider breaking the API to optimize for people using it correctly with these new requirements, since no one is listed as depending on it at the moment.

I suspect the easiest way to implement this will be to put all of this into a bitfield in an atomic u32.

It can be a starting point if @mitsuhiko is up for it though it won't be hard to do something similar.

EDIT: Also seems like we could learn some lessons from termcolor

[epage]

Just read up on the new feature resolver (blog post, edition guide). It sounds like firestorm is not impacted and that we could do something similar. All libs could depend on a no-op version of the CLI color control crate. An application can then enable the features it wants. I think a middle ground would also be acceptable, libs could choose to depend on the default policy if they offer it through a color feature flag, whether its default or not.

[uazu]

Remember that terminfo and termcap came from a time when there were a lot of very weird non-VT-series terminals around, and if you wanted your app to work at all, then you had to use the peculiar sequences required by those terminals. Nowadays it's reasonable to assume basic ANSI support. So, for colour if you just use plain 8-colour ANSI sequences and maybe bold, it shouldn't need any feature testing. So either aim for the lowest common denominator (easy) or try to get fancy and detect extra features (harder). Also bear in mind that the user might be running with either a white or black default background (colour 49). So blue on default-background might be readable on white, but probably won't be on black. This is all UNIX, though -- I don't know the Windows situation.

[epage]

I now have a first pass of concolor-control. I tried to model the detection logic off of termcolor as much as possible. Creating a PoC PR for termcolor will be the real test for how well it works.

EDIT: termcolor PoC

[pksunkara]

Looked at your code and it looks good. One concern I have is about allowing ColorChoice for https://docs.rs/concolor-control/0.0.6/concolor_control/fn.set.html instead of a bool. This api feature should be about programmatic forcing of colors.

[epage]

We need to handle the CLI case, like concolor-clap. The developer needs to the ability to say "do what you want".

• This could be handled by just not calling set
  • I try to avoid APIs where a state can't be unset, usually regretting that constraint
• This could be handled by the developer doing a get
  • This is adding extra boilerplate that an enum resolves

On top of those

• The Rust API Guidelines encourage using enums over bools to clarify intent
• None of these cases help with AlwaysAnsi vs Always for styling still supporting pre-ANSI coloring on Windows.