Skipping joiners during GSUB

[adrianwong]

If a ZWJ is placed between two characters with the intent that the two should ligate, but a font has not provided a lookup that uses the ZWJ, should the ZWJ be skipped in order to allow the ligation?

Currently, Allsorts doesn't skip the ZWJ. Here is an example using Noto Serif:

Likewise, should joiners (both ZWJ and ZWNJ) be skipped in backtrack/lookahead sequences?

Allsorts doesn't do this either. Using Noto Serif again, this is a chaining contextual lookup where "i" is the input, and the tilde is the lookahead:

HarfBuzz skips the joiners in both cases. Relevant links here and here. Should Allsorts follow suit?

[behdad]

Obviously I think what we are doing in HarfBuzz is the best way to interpret Unicode with OpenType. I've made all those decisions with @jfkthame.

[adrianwong]

Thanks for your input, @behdad. Skipping ZWJs seems like the sensible thing to do, as one would expect that a ZWJ should not block ligation.

However, can you elaborate on why skipping ZWNJs in backtrack/lookahead sequences is "the right thing to do" and "that backtrack/lookahead should match the actual glyphs; presence or lack thereof joiners should not affect what matches"? I'm referencing your comments in the links I shared, and am curious as to how you (or the both of you) arrived at that conclusion.

Perhaps this behaviour should be documented the OpenType shaping docs, if it isn't documented anywhere else (cc @n8willis).

[mikeday]

I'm also curious @behdad would you happen to have a good example of how joiners might be used in practice that could interfere with the contextual lookup process?

The example of f + zwj + i seems very reasonable but placing a zwnj between base and mark is more of a contrived example to demonstrate the shaping behaviour; is there a situation where something like this could actually happen in Arabic text?

[behdad]

Here. A purely demonstrative example:

Imagine you have a rule that ligates C,D but only if preceded by A,B glyphs. That is, A,B,[C,D] is the sequence with A,B part being backtrack.

Now if input is A,ZWNJ,B,C,D, we STILL want the rule to match. That's why we ignore ZWNJ in backtrack/lookahead.

[mikeday]

Thanks that seems reasonable, perhaps we can scan some fonts for suitable ligatures and find real examples of that case.