Rewrite

[arthurkehrwald]

This started out with the goal of replacing the legacy gRPC implementation with the new grpc-dotnet and gradually turned into a full rewrite. I have tested this in the editor and in builds on Windows 11, Ubuntu 24 and macOS 15. I have not tested on Apple silicon.

Improvements

• Replace legacy dependencies
  • Moved to grpc-dotnet
  • Replaced legacy IMGUI inspector with UI Toolkit UI
• User experience
  • Asynchonous everything. Doesn't ever block the main thread.
  • Warns about any mismatches in the machine description on startup
  • Tooltips for all inspector options
  • Includes MPF binaries for Windows, Linux and macOS in the Unity editor and in builds. Builds only include the binaries for the target platform.
  • Using a local MPF installation is still possible, as well as not starting MPF at all and using an already running version of MPF for debugging purposes.
  • Supports opening MPF in a separate terminal window in macOS and Linux. Previously, there was no window on those platforms and MPF would crash unless run in text mode.
  • After installation, creates a sample machine folder in the streaming assets path of the Unity project. This folder is also used as the default machine folder of the MpfGamelogicEngine component.
  • The state of the MPF process is displayed in the inspector.
• Reliability
  • Kills any MPF processes on startup and shutdown.
  • On startup, pings MPF until it responds instead of waiting for 1.5 seconds and then assuming it is ready. This means the game can start as soon as MPF is ready and startup doesn't fail if MPF is too slow. There is still a customizable timeout (10 seconds by default) so we can at least show an error to the user and exit the table once we have a player app. I created a PR in the main MPF repo for the necessary changes on their end: Update VPE dependencies and rebuild interface missionpinball/mpf#1865 In the meantime, official versions of MPF are supported using the same method as before.
  • Doesn't leak memory or leave processes running if things go wrong.
• Structure and readability
  • The code is no longer divided into a compiled library and a UPM package. The entire thing is a UPM package.
  • The logical structure is also simplified. There are only two main classes: MpfGamelogicEngine does what we actually care about. It sends switch changes to MPF and applies MPF commands to the playfield. MpfWrangler takes care of all the tedious details involved in starting up, communicating with, and shutting down MPF.
  • Everthings runs on the main thread.
  • The state of the MPF process is clearly defined: Not connected, starting, connected or stopping.

Reverse improvements

• Because Unity regenerates solution files all the time, we cannot use Grpc-Tools to compile the proto file. This must now be done manually whenever it updates.
• There are no more coil, light, and switch icons in the new inspector. I couldn't be bothered, sorry.

Requirements

• Protobuf
• Grpc.Net.Client and its dependencies
• YetAnotherHttpHandler and its dependencies

I installed the first two using UnityNuGet and the last one straight from GitHub via URL in the Unity Package Manager. These dependencies must currently be installed manually because the NuGet packages require a scoped registry and Unity package manifests are not allowed to pull dependencies from a git URL.

[arthurkehrwald]

Alright looks like builds are fixed. That leaves publish and release.

[arthurkehrwald]

And yetanotherhttphandler is still missing of course. Did you find a way to mirror it? I think just making it a git submodule would also work. A bit wonky maybe.

[freezy]

Lemme check!

[freezy]

Okay so this is less trivial than I thought. With Verdaccio, the NPM registry we're using, you can easily uplink to other registries, scoped by namespace, but this isn't published on any NPM registry, only on GitHub. So we'd need to find some kind of bridge plugin for Verdaccio that uplinks a GitHub link to a namespace.

I'll do some more digging, but if submodules are a way (or simply download it from GitHub during build?), maybe it's worth exploring.

[arthurkehrwald]

I'm working on downloading yetanotherhttphandler from github during the build process as you suggested. I'm wondering why we are copying things into a tmp folder before uploading in the build action. This is done in the main repo and in pinmame as well:

- run: |
    mkdir tmp
    cp -r VisualPinball.Engine.PinMAME.Unity/Plugins/${{ matrix.rid }} tmp
- uses: actions/upload-artifact@v3
  with:
    name: Plugins
    path: tmp

Why not just upload the folder directly?

[jsm174]

- run: |
    mkdir tmp
    cp -r VisualPinball.Engine.PinMAME.Unity/Plugins/${{ matrix.rid }} tmp
- uses: actions/upload-artifact@v3
  with:
    name: Plugins
    path: tmp

Why not just upload the folder directly?

At the time, we were leveraging a feature where artifacts from different jobs could be uploaded with the same name. The artifact download would then be a single zip with subfolders for each architecture:

If we download and expand one of the packages from the registry you'll see this layout:

Anyway, the actions do need a bit of updating as things have changed. In v4 of upload / download artifact, you can no longer use the same artifact name:

https://github.com/actions/upload-artifact?tab=readme-ov-file#v4---whats-new

We'll have to look into:

There is also a new sub-action, actions/upload-artifact/merge. For more info, check out that action's README.

But you're right. You could just copy directly.

[arthurkehrwald]

Ok thanks for the explanation. I'm already using the merge action. But it seems like that is also not the best solution:

For most cases, this may not be the most efficient solution. See the migration docs on how to download multiple artifacts to the same directory on a runner. This action should only be necessary for cases where multiple artifacts will need to be downloaded outside the runner environment, like downloads via the UI or REST API.

[arthurkehrwald]

Yetanotherhttphandler is now downloaded as part of the dotnet build process. When run locally, that's all there is to it. For deployment, I disable the download of yetanotherhttphandler during the platform specific build processes, because it would just download the same thing four times. It is instead downloaded once in a separate job and uploaded. I have no idea whether publishing actually works though. Any way to test that or should we just merge and see what happens?

[arthurkehrwald]

I also switched back to the official https://github.com/actions/download-artifact from https://github.com/dawidd6/action-download-artifact, since downloading artifacts from another run is now supported: actions/download-artifact#3 (comment)

I assume that was the reason why you used https://github.com/dawidd6/action-download-artifact.

[arthurkehrwald]

actions/download-artifact v4 also supports merging multiple artifacts now, so we shouldn't need to merge the uploads manually using https://github.com/actions/upload-artifact/blob/main/merge/README.md

[jsm174]

I also switched back to the official https://github.com/actions/download-artifact from https://github.com/dawidd6/action-download-artifact, since downloading artifacts from another run is now supported: actions/download-artifact#3 (comment)

I assume that was the reason why you used https://github.com/dawidd6/action-download-artifact.

Awesome! Correct on dawidd6/action-download-artifact.

BTW, really fantastic work on this project!

[arthurkehrwald]

BTW, really fantastic work on this project!

Thanks!

[freezy]

Let's merge and see what happens :)

[arthurkehrwald]

I was about to update the readme and just realized I forgot about the .meta files of the binaries. It says in the wiki they are important to 'avoid conflicts', but I don't get why we can't just .gitignore the entire dependencies folder and let Unity regenerate them.

[freezy]

Yes, new PR then. @jsm174 looks like an auth error?

[freezy]

The thing with the .meta files is that they actually contain relevant config, and sometimes the default config when it's regenerated isn't what we want.

[jsm174]

yeh the meta files have the architecture flags which I remember we're a pain to get configured properly.

[jsm174]

Yes, new PR then. @jsm174 looks like an auth error?

Strange that is needs creds for https://registry.npmjs.org

Maybe its trying to pull or get deps?

echo "//registry.visualpinball.org/:_authToken=${NPM_TOKEN}" > ~/.npmrc

[freezy]

I recently restarted Verdaccio (had 1500+ days of uptime, lol), but the config and all that stuff should be persisted, since it's on a separate volume.

[arthurkehrwald]

Maybe we try this: https://github.com/actions/setup-node

[arthurkehrwald]

Maybe we try this: actions/setup-node

PR: #35

[arthurkehrwald]

I forgot to mention, to not block the main thread during initialization, I made MpfGamelogicEngine.Init asynchronous, so we need to merge freezy/VisualPinball.Engine#504 for this to compile.