Colour handling improvements

[Jermolene]

Introduction

This PR brings several new features for end users:

• Automatically switching between a dark and light palette as the operating system setting changes (and to do so without making the wiki dirty)
• Customisation options for palettes. For example, users might choose a base hue, with the colours of the palette automatically adapting to it
• A generalisation of the dark vs. light mechanism to allow an arbitrary number of distinct schemes that are dynamically selected. For example, a palette that has a different scheme for night, morning, day and evening that automatically change with the time of day

There are also new capabilities for palette authors:

• Inheritance for palettes, making it easy to create chains of variants of a base palette
• Self contained palettes that can contain both dark and light variants (or variants for any other custom scheme)

To make all of these new features possible, this PR also includes some useful new general purpose mechanisms and features:

• Background actions that are triggered whenever there is a change to the results of a specified filter
• Several new filter operators for manipulating colour values. The underlying functionality comes from the color.js library
• New media query tracking mechanism that can track the results of any CSS media query (not just dark mode), storing the results in a shadow $:/info/... tiddler
• New changecount filter operator
• New :apply filter run prefix

Try the Demo

A preview build is available at https://deploy-preview-8702--tiddlywiki-previews.netlify.app/

Changes to Palettes

Palette entries are now defined as filters that must be evaluated, rather than wikitext that must be wikified.

This makes it possible to create palettes that reference and modify other colours. For example:

tiddler-controls-foreground-selected: [tf.interpolate-colours[background],[foreground],[0.9]]

Retrospectively re-interpreting palettes entries as filters would normally mean that <<colour background>> would no longer work. Instead these entries are automatically converted to the updated form [function[colour],[background]].

Palette Compilation

The key idea underpinning these changes is a fundamental change to the way that TiddlyWiki handles palettes. At the moment, palette entries are named items that can contain either a CSS colour string, a CSS colour token like "inherit", or can use the <<colour>> macro to reference another colour palette entry. Thus, palette entries have to be wikified before they can be used. This has turned out to be very limiting and doesn't provide a viable path to the complex colour manipulations shown above. Switching to filters might make things worse, by encouraging authors to use complex expressions within palettes.

The fix is compiled palettes: at the point of switching to a new palette, the colours within it are "compiled" to raw CSS colour values (typically but not necessarily in #rrggbbaa format). This allows palette entries to be used directly, without the requirement to wikify them.

The static palette is created in a new system tiddler $:/temp/palette-colours by an action procedure that is invoked at startup and when switching to a new palette.

There should not be any backwards compatibility issues because the use of background actions means that any code that changes $:/palette will automatically trigger the recompilation of the palette.

This change also allows us to change the <<colour>> procedure to be a function, which allows it to be used as the value for a style attribute:

<div style.background=<<colour tiddler-background>>>

Automatic Palette Readability Tests

Palettes can opt to include readability tests as special palette entries. The results of these tests are shown at the bottom of the palette chooser. For example:

?base-background-ink: [tf.check-colour-contrast[base-background],[base-ink],[45]]

The test framework looks for palette entries starting with a question mark and runs the associated filters. The filter should return nothing if there are no errors, or a textual error message if the conditions are violated. Sample output:

alert-contrast: 42.357: alert-background/foreground contrast is too low
background-foreground-contrast: 41.915: background/foreground contrast is too low
base-ink-secondary: 42.357: base-ink/base-secondary contrast is too low
base-paper-ink: 41.915: base-paper/base-ink contrast is too low
base-paper-primary: 32.682: base-paper/base-primary contrast is too low
base-paper-tertiary: 30.502: base-paper/base-tertiary contrast is too low
code-contrast: 28.565: code-background/code-foreground contrast is too low

Colour Manipulation

We need a colour manipulation library that can calculate variants of colours. Only color.js met the requirements of being able to work with P3 colours and the OKLCH colour space. It also includes a CSS colour string parser which can replace the simple one that TiddlyWiki has always incorporated.

Media Query Tracker

The CSS media query tracker allows a media query to be bound to an info tiddler so that the current state of the query is reflected in the value of the tiddler. The value is updated dynamically.

The use of the info mechanism for the CSS media query tracker means that these tiddlers are dynamically created as shadow tiddlers within the $:/temp/info plugin, and so do not appear in tiddler lists.

The mechanism is used to implement the existing dark mode tiddler with the following configuration:

title: $:/core/wiki/config/MediaQueryTrackers/DarkLightSwitch
tags: $:/tags/MediaQueryTracker
media-query: (prefers-color-scheme: dark)
info-tiddler: $:/info/browser/darkmode
info-tiddler-alt: $:/info/darkmode

Note the use of info-tiddler-alt to specify a redundant secondary info tiddler. This is used by the dark mode tracker to maintain compatibility while changing the info tiddler title for consistency.

Background Actions

The new background actions mechanism allows action strings to be invoked automatically in the background whenever the results of a filter change.

The preview includes a demonstration background action that displays an alert with a list of tiddlers in the story river whenever the story river changes:

title: SampleBackgroundAction: Story Change
tags: $:/tags/BackgroundAction
track-filter: [list[$:/StoryList]]

<$action-sendmessage $message="tm-notify" $param="SampleBackgroundAction: Story Change" list={{$:/StoryList!!list}}/>

Story List:

<ol>
<$list filter="[enlist<list>]">
<li>
<$text text=<<currentTiddler>>/>
</li>
</$list>
</ol>

apply Filter Run Prefix

An experimental new filter run prefix that makes it possible to use computed values as variables within a filter run. For example:

\function tf.interpolate-colours(paletteEntryA,paletteEntryB,weight)
[function[colour],<paletteEntryA>] [function[colour],<paletteEntryB>] :apply[<weight>colour-interpolate:oklch<$1>,<$2>]
\end tf.interpolate-colours

Backwards Compatibility

• The current content of $:/PaletteManager is moved into $:/PaletteEditor, and $:/PaletteManager repurposed as the control panel palette switcher
• $:/config/DefaultColourMappings/ now only supports CSS colours, and not indirections via <<colour X>> or [function[colour],[X]]

References

• The media query tracker and background actions mechanisms were originally developed in a separate pull request
• [IDEA] Tools for manipulating colours and palettes #5236 discusses the general idea of dynamically computed colours, and includes a very neat demo from @cdruan
• A thread discussing using the colour macro as an attribute value - https://talk.tiddlywiki.org/t/how-to-use-the-colour-macro-as-an-attribute-value/10051
• A thread discussing improvements to colour handling, including a similar idea for palette previews - https://talk.tiddlywiki.org/t/idea-approach-to-palettes-and-themes/9431

Progress

• Documentation
  • Palettes and colour schemes
  • Media Query Tracker
  • Background actions
  • changecount operator
  • colour-interpolate operator
  • colour-lighten operator
  • colour-darken operator
  • colour-get-oklch operator
  • colour-set-oklch operator
  • colour-contrast operator
  • colour-best-contrast operator

[github-actions]

Confirmed: Jermolene has already signed the Contributor License Agreement (see contributing.md)

[kookma]

Is there a demo to test colour handling improvement?

[Jermolene]

Is there a demo to test colour handling improvement?

Hi @kookma I've posted a preview to tiddlyhost

[Jermolene]

Very promising developments. The "ink" and "paper" metaphors are intuitive, and interpolation effects seem to be working well!

Thanks @springerspandrel

Sidebar tabs are not yet "inheriting" the tab-background and tab-background-selected palette values in TwentyTwenties (though it seems that's the intention of having these default to [function[colour],[tab-background]] (etc.)

That should be fixed now.

(a) tab-background and tab-background-selected colors don't get any preview even though sidebar tabs are very much part of "out-of-box" GUI... Having a mock set of sidebar tab-rectangles (one active, one or two inactive) might be helpful, at the risk of more visual noise... (Meanwhile, we do currently get a preview of search window input-box — despite duplicating tiddler background for all bundled palettes except Solarized Dark.)

The previews now show sidebar tabs in the appropriate colours.

(b) Selecting new primary color doesn't affect palette's preview-thumbnail at all.

That should be fixed now.

(ONE solution: Include minimalist mock-preview of sidebar tab list content — so the tiddler-link color appears in preview, whether or not a palette has distinct value for site-title-foreground ... ALTERNATE solution: have some primary link-color "accents" within the top mock tiddler such as HelloThere — noting that the real HelloThere conveniently does feature several visible links within a short tiddler body!).

Both suggestions are now implemented.

(c) Selecting new secondary and/or tertiary color choices doesn't affect palette preview

That should be fixed now.

and user also lacks obvious places — in larger interface — to recognize dynamic effects of that selection. If the second "mock" tiddler in preview-thumbnail were to include a small block of code, that would nicely gives a hint of tertiary.

A block of code is an excellent idea.

(Including a natural preview of the secondary color seems more difficult, since having an alert within preview thumbnail is awkward.)

I have included an alert in the latest update, but the challenge is that it obscures the title of the first tiddler. Perhaps previews need multiple pages to show things like alerts, notifications and modals.

[pmario]

Indentation please.

[Jermolene]

Just to be clear, this file hasn't been changed (yet), it is just moved from $:/PaletteManager

[pmario]

ah ok

[Jermolene]

I meant to add that this file needs substantial revision so I will add indentation when I do that.