Make the normalizer work with new Unicode 16 normalization behaviors

[hsivonen]

Closes #4859

ICU4C PR coming up.

[hsivonen]

ICU4C PR: unicode-org/icu#2994

[hsivonen]

This reverts #4538 , which turned out to be a bad idea.

[hsivonen]

I've tested this with Unicode 16 data, but this PR doesn't include the new data.

[sffc]

• @hsivonen - PR Make the normalizer work with new Unicode 16 normalization behaviors #4860 adds compatibility with Unicode 16, but it potentially regresses performance on C forms (not NFD or NFKD) due to additional branches, but this hasn't been benchmarked. PR Adapt the collator to Unicode 16 normalization changes #4878 should not have any performance impact.
• @echeran - I think we should keep Unicode 16 changes together.
• @sffc - What types of characters hit the branch?
• @hsivonen - It impacts the performance of the pass-through operation for NFC.
• @sffc - It seems not super urgent to lang these since Unicode 16 isn't out yet, but we should do the work to test and benchmark them.
• @hsivonen - We would want a long string of CJ text and want that to go fast.

Conclusion: spend the time to benchmark these changes, in conjunction with #4967. Do this before 2.0 because it might involve a data struct change.

[hsivonen]

We should merge this and #4878 now regardless of the outcome of #4967 . If the outcome of that investigation shows that it makes sense to rearrange the bits, let's land that change on top of this one.

[Manishearth]

Yes, this seems like a minor optimization at best. Let's land this, we should have 16 support for 2.0