core: Fix some issues with hitTest

[Herschel]

Fixes some issues with our winding # calculation which would cause incorrect results for hitTest.

• The convention for handling an intersection at endpoints was not the same between lines and bezier curves.
• The bezier curve winding # function was not properly handling some cases where the curve was strictly y-monotonic.
• Simplify the code a bit so that ray-curve intersections are returned in a consistent order based on upward/downward crossing.

[Herschel]

I'll add a test case for this tomorrow; this could use some testing as games are very sensitive to changes here.

@Aaron1011 Could you check if this solves the issue in Wonderputt?
@Toad06 Could you check if there are any regressions, particularly in platform games like Ultimate Flash Sonic?

[Aaron1011]

@Herschel While this branch does fix the specific point tested in my SWF, it unfortunately doesn't seem to resolve the full issue. By moving my mouse around inside the largest empty region, I was able to generate many more incorrect 'hit' points. I've updated https://github.com/Aaron1011/ruffle/tree/hittest-debug to test for these points by default, and to mark each with a purple pixel. Running the SWF in Flash Player traces 'false' for all of those points, while running in on this branch traces 'true' for all of them except the first one (which was fixed by this PR).

[torokati44]

(FWIW this also doesn't fix #4064 yet.)

[Toad06]

This fixes an issue in Steppenwolf 3-1. It was impossible to hide behind the tunnel without jumping:

steppen.mp4

---

In Ultimate Flash Sonic, the character seems to get stuck less often. But I also noticed a few regressions:

sonic1.mp4

sonic2.mp4

---

The game in #5481 (which is not a platformer) doesn't seem to like these changes, as unexpected collisions happen more often even when the file is displayed in its natural dimensions.

[Aaron1011]

Wonderputt holes 1 and 14 no longer have false-positive hits with this PR.

[Herschel]

Thanks for testing! I checked with all of the above cases and it seems okay. I also added the Wonderputt shape to the hitTest test.

@Toad06 could you give this a bit more testing?

[Herschel]

Looks like #4064 still has some issues.

[Herschel]

Looking at #4064, it actually seems to be a case of #5492, where the ground being hitTested is the ground from level1 even on level2. I was able to hack in a fix on the SWF to re-get _root.game.map.ground and was then able to complete level 2 successfully.

[Toad06]

No regression found. 👍

This also fixes Monkey Lander, NoName Game (#5481), NoName Game 2 which were impossible to complete.