Rework mod format, and futureproof it (2024 edition)

[EM4Volts]

up there is a flowchart of a concept i worked out in about an hour with gecko.

i had a long conversation with gecko yesterday about mod related stuff
such as:

• future of mods/packages
• how mod managers handle the both
• coremods
• mod managers in general
• supporting new/other platforms

The result of this entire conversation is a small flowchart. Remember, this is very wip
The goal of this entire thing is to finally standardize, and properly document the entire mod format and make it future proof in case a platform has to be switched.

feedback is highly appreciated

Replaces #472

[EM4Volts]

another note we had on this is to potentially remove "enabledmods.json" and put an .enabled .disabled file into the root of a mod or into the mod.json to make sure northstar associates the disable/enable with the right mod, instead of by name

[Jan200101]

.enabled seems redundant as the lack of a .disabled file should signify that its enabled.
This could also be added to the modinfo, but may be overkill.

[GeckoEidechse]

For reference, the disabled file is how IcePick does mod disabling.

[EM4Volts]

no, the modinfo file is a file that is strictly for mod managers to store user information on the mod to make them able to handle updating, renaming etc.
however adding an "enabled" field to the mod.json would be something i guess could work

[Jan200101]

The topic of how plugins should be handled came up and I wanted to share my 2 cents (as one of the few plugin creators):

• plugins should share the same space as mods to simplify how they are managed by mod managers
• its unclear how plugins will be identified. Just look for plugin.dll? have it be specified as a name in modinfo as a nullable field? Use the plugin name?

Also something else I would love to see:

• specifying dependencies so Northstar can load them ahead of time or prevent loading a mod with missing dependencies
  • on that note, maybe optional dependencies too? It would be useful for mod managers to give users a greater choice.

[EM4Volts]

yeah, we didnt quite know how we would identify plugins. gecko said we could just look for a dll, my thought was that plugin devs could just add a plugin bool in the mod.json to make it identifiable

[ASpoonPlaysGames]

Plugins could just be part of a mod.json if we are going to make them share a space?

[F1F7Y]

> Reworking mod handling
> Mentions very little about the structure of the mods and the format of what is currently mod.json
> Has a giant ass graph

To be at least a bit helpful imo this discussion should be based around how the game handles/shoud handle loading/unloading/downloading of assets.

Offloading work to modmanagers also means you'll have manager specific behaviour and be dependent on managers implementing a feature which isn't desired.

[EM4Volts]

because we want to seperate the files mod managers manipulate and the one the launcher manipulates/needs
if you want to change mod.json u need to give proposals

[F1F7Y]

%profiedir%              <- search path so engine can find mod.vdf
  mods
    %modname%            <- search path so cache can find the modded files
      mod.vdf            <- we already have keyvalues mapped out, no extra work
      manager-info.json  <- if you really want to save some info (not much worth saving imo)
      .disabled          <- indicates the mod is disabled
      scripts
        scripts.rson
        %map_name%.rson  <- would have the same placement as scripts.rson
      resources
      vpk
        vpk.rson         <- probably not needed
      rpak
        rpak.rson
      audio
        audio.rson
      %pluginname%.dll
      libs/
  ...

Probably forgot about some things, but this would be a pretty good step forward imo.

mod.vdf would only have generic information (name, description, version, ...) The information about the modded files would be scattered into .rson files unlike the current mod system where some things are in mod.json and some in their own json files with no file structure.

I don't see much value in manager-info.json as the only worth while information without creating duplicates is the site its was downloaded from. And if you're letting the modmanager handle downloading it shouldn't matter where it got the mod from. Could also give the user the choice to pick a site if the mod is on multiple sites like gnome software does.

%modname% should probably be standardised to AUTHOR.MODNAME.MAJOR.MINOR.PATCH, but good luck doing that if you want multiple "official" sites.

[uniboi]

Whatever the new mod system is going to be it should be consistent.
Right now there is a split between a bunch of random things that are in the mod.json and other configurations that got created as time passed.

.
└── Northstar.Custom/
    ├── audio/
    │   └── pilot_grapple.json <-- configuration for this specific event
    ├── mod/
    │   ├── scripts/
    │   │   └── vscripts
    │   └── vpk/
    │       └── vpk.json <-- only used for aliasing and sideloading vpks, still needed in the new format
    └── mod.json <-- has scripts, convars, init scripts, mod info, dependency constants and more

On top barely anyone knows that files in the mod/ folder override files from the game, if that's even still a thing because things like that are documented nowhere.

So either put all configuration into the mod.json or like fifty suggested split things where they make sense into a scripts.rson, vpk.rson, sound.rson, pak.rson etc.
In an ideal world mod.json only contains metadata about the mod like it's name. No init scripts, no convars and no scripts.

Also I think the file override model should get reworked entirely, but that complicates things further with the benefit of added security and mod compatibility if done correctly.

Some ideas for compat would be

• Add a listener system for global functions that allows callbacks and replacing the original script function with a replacement for every listener each

  void functionref(entity player) ornull Listener_CodeCallback_OnPlayerRespawned( entity player )
  {
    print( player.GetPlayerName(), "respawned" )
    return null // if null is returned, the original CodeCallback_OnPlayerRespawned is going to run (when another listener doesn't block it)
    return void function(entity) {} // if a functionref is returned, it's going to run instead of the  original script function. If multiple listeners return a replacement all of them will be ran.
  }
  
  void function MyMod_Init()
  {
    AddListener(" CodeCallback_OnPlayerRespawned", Listener_CodeCallback_OnPlayerRespawned )
  }

  That's going to be complex to implement and introduce some overhead to sq_call which can bog performance. Better would be a new opcode but that's a serious amount of work

• Over time Rewrite vanilla scripts with an API that accommodates everything mods might need. That's going to take absurd time and endless arguments over apis

The unfortunate truth is that overriding files is a huge headache but unfortunately for some mods unavoidable.

[GeckoEidechse]

Splitting some stuff into scripts.rson, vpk.rson, etc definitely sounds compelling. We should definitely have an API version field or something in the mod.json so that we know which "version" the mod is, so that can have some temporary legacy support for mods that follow the old structure.

For scripts replacing individual functions instead of files would be super dope (I think this has been sought after forever now), especially cause that would also make mods that modify larger Squirrel files file mainmenu.nut more resilient to updates. If anyone would like to work on that, please LMK <3

[Alystrasz]

We should write a RFC for this unified mod format (UMF) 😃

[pg9182]

Can we please keep the mod directory read-only (i.e., don't create .disabled files in there automatically)? This makes it much easier for mod managers and other tooling to work with.

Also, adding an env var to specify additional mod search dirs (PATH-esque, earlier takes precedence and overrides later ones with the same name) would be useful for the dedicated server.

[pg9182]

And a potentially controversial idea about enabling/disabling mods: disable everything by default, then have various methods (convars for mod settings ui, server required mods, env var, cli arg) to enable them, and also a blacklist which takes precedence.

This prevents multiple things from fighting over a single enabled/disabled mechanism, makes it clear why a mod is enabled if it's enabled, is flexible and future-proof, keeps the mod directory read-only, and is easy to manipulate externally if required.

For dedicated servers, probably enable all mods by default, since it doesn't make much sense to do otherwise?