Fully automated end2end testing

[vorburger]

It could be fun if our (new, introduced yesterday) manual testing steps could be fully automated.

This would be possible with a programmatic Minecraft client , such as:

• https://github.com/willemml/hl-mc-kt (unmaintained, take over?)
• https://github.com/PrismarineJS/mineflayer (JS)
• https://github.com/Defective4/Another-Minecraft-Chat-Client (part of, refactor)
• https://gitlab.bixilon.de/bixilon/minosoft (part of, refactor)

https://github.com/GeyserMC/MCProtocolLib/blob/master/example/com/github/steveice10/mc/protocol/test/MinecraftProtocolTest.java looks great, but is too low level. Ask them if they know of any higher level libs built on them?

[vorburger]

We concluded that https://github.com/PrismarineJS/mineflayer is the way to go here! The (future) "test" then doesn't need Java, nor TestContainers, nor JBang - we'll just add 1 line to ./test which launches node run mineflayer (or whatever). (Later split off a new ./start [only] called by ./test.)

[vorburger]

#348 made a start for this. What's left:

1. Run Mineflayer test on CI, instead of just running Dockerfile for each PR; see Add cloudbuild.yaml (re. #327) #355 et al.
2. factor out a start & stop from test
3. moar tests (if possible?)

[vorburger]

1. will be taken care of in Run build in container - but still cache (fixes #374) #381 + Support docker AND podman (just via podman-docker) #375 + Run full ./test and just ./make for every PR on GCB #382

2. doesn't seem that important anymore (to me)

3. is what we should still do ASAP... the current test just makes sure a Minecraft user can login, but doesn't actually test any of our plugin's functionality; the current test as-is would still pass as long as the server started, even if our plugin was completely broken. The goal here should be to automate at least some sort of subset of what we are describing in our Testing doc.

[vorburger]

@edewit I just thought about this again... especially with #375 now done, which will then enable #382, it would be REALLY cool if we could slightly beef up our Mineflyer based test to do something a bit more interesting than only login - anything that could test our plugin loaded and works. That could give us the confidence among other things e.g. to approve Dependabot PRs, among other things, if we see them pass the build, once that includes a more useful real test. If showing title and make entities narrate isn't something that we can easily test with Mineflyer, then let's add whatever else is needed to our test JS and/or Java that we can actually really test with Mineflyer? Have a go at it if you are up for it! 😸

[edewit]

Having entries narrate in a mineflyer test is not possible, but we would be able to get the chat, so let me add that

[vorburger]

@edewit with #389 merged, shall we say that fixed this, and close this issue now?

Or can / should we do more here? You asked if we could make the Mineflayer Player OP so that /say works - but do we need that? Now that we have #389, are we missing anything useful? Remember we looked at that a few weeks ago, but because we don't know the UUID that Mineflayer uses, we couldn't find an easy way to add it to the OPS list in ./test.

[edewit]

Right let's close this even though it doesn't test if 'say' gets executed it does test if the scripts work as it see the title being displayed. The only thing for fully e2e test would be if we test if an upload and reload would also succeed

[vorburger]

#400 adds additional coverage.