Please make it for linux

[JoaoVitorSCS]

Please make it for linux, so it would be possible for us to play games without damn workarounds that all of a sudden stops working

[Al12rs]

Usvfs, the virtual file system Mo2 uses, hooks windows filesystem functions to change their behavior. It can't be "ported". Someone would have to create a VFS specifically designed for Linux and make it work for Mo2.

It was estimated that a working port of Mo2 would require multiple years of work of low level Linux knowledgeable developers. None of the active developers is interested Linux or provind support for it.
We are happy to welcome new developers but if their intention is to make a Linux version we would warn them it's not work their time

That said a lot of people have managed to get Mo2 to work on Wine. For more information you can check the thread we kept open for it. We don't support it but if it works all the better.

#372

Hope you can work something out there

[VelorumS]

Someone would have to create a VFS specifically designed for Linux and make it work for Mo2.

@Al12rs What are the difficulties with making Linux overlayfs work with Mo2?

[Al12rs]

Someone familiar with overlayFs should rewrite all the parts of Mo2 that interact with usvfs to instead make use of overlayFs, and it might not be so easy since usvfs is a dll you inject into a target application and it shows the virtual folder to the application only, while overlayFs would be an always on thing that might instead need activation and deactivation.

So some work there. This is assuming that overlayfs does semantically what we want it to do and doesn't have limitations, so maybe some tweaking there. Then hopefully the games all work fine over an overlay.

Then you would have to change all the other windows specific code of Mo2 since it does have more than some since it was decided to not support Linux.

But from what I know none of the current developers neither has the time or the interest in supporting Wine, let alone native Linux support. Wine is needed anyways for a lot of games so it seems like a waste of time.

We are currently short on free developers, we have trouble keeping up on normal maintainance so considering native Linux support is not in the cards.

[AnyOldName3]

It's multiple layers rather than just one, so I don't think you could use overlayfs directly. It shouldn't be too hard to make a USVFS equivalent with FUSE, though, as it's something more directly supported by the OS and Unix filesystems are an awful lot simpler than Windows ones. I made a (too slow to be usable) clone of USVFS using Dokan.NET as a university project, I actually don't think that would be the biggest part of getting MO2 to work on Linux.

Other parts include:

• Getting rid of all the bits of MO2 that assume they're running on Windows. We use cross-platform dependencies, but lots of places directly use the Win32 API or assume that dynamic libraries are .dll files, or path separators are backslashes etc. I'd expect this to be where the bulk of the work is.
• Make it so different VFS implementations can be swapped in. This shouldn't be insurmountable as USVFS is more separate than the old hook.dll was, and it's useful even without a full Linux port as KxVFS could be used, too.

[VelorumS]

it shows the virtual folder to the application only, while overlayFs would be an always on thing that might instead need activation and deactivation.

That's fine, Linux namespaces do that: a process can have mounts that are visible only to that process.

It's multiple layers rather than just one, so I don't think you could use overlayfs directly.

Can use directly if changed a constant in Linux kernel (allowed overlay depth) and recompiled the kernel. Non practical for a direct one (read further for another option with overlayfs).

I actually don't think that would be the biggest part of getting MO2 to work on Linux.

That's what I've noticed.

Getting rid of all the bits of MO2 that assume they're running on Windows. We use cross-platform dependencies, but lots of places directly use the Win32 API

Uses a bit of file dialogs, Windows registry, system information, process creation, taskbar, access rights...

Questions:

What's the https://github.com/ModOrganizer2/modorganizer-archive project? It has an innocent description "a wrapper round the 7zip" but tones of WinAPI/COM stuff inside?

Is there a difficulty with a virtual filesystem implementation that constructs one layer somewhere (and reconstructs it as needed) to give that to overlayfs?

[Al12rs]

@ChipmunkV It might be easier and faster to discuss things on discord rather than this closed issue. Here is a link: https://discord.gg/QuGJkRK

The archive library is something Mo2 uses to extract downloaded mods, or do stuff like listing the files inside an archive I guess.
For the second question I'm not too familiar with Linux vfs and I'm not even sure of what you are referring to with layer in this case. In any case I would think it better to continue the conversation on discord.

[VelorumS]

I'll start porting. Projects are here: https://github.com/stars/ChipmunkV/lists/mod-organizer-2

[Al12rs]

Do you think you could join discord so we can chat about this? If you are serious about this we are open to provide info and help as you go along. I would much prefer if you were as informed as possible going in and keeping an open discussion with us.
https://discord.gg/QuGJkRK

[jthistle]

There's already a 'port' of sorts - it's a script that installs mo2 into the Wine compatability layer for Linux. See https://github.com/rockerbacon/modorganizer2-linux-installer.

As far as I've tested it, it mostly works.

[VelorumS]

@jthistle that's interesting, I was only aware of the "steamtinkerlaunch" effort to simplify an install of MO2 in Wine.

I would much prefer if you were as informed as possible going in

I've messed it up, I'm already a couple of weeks into code.

See you in discord eventually.

[Al12rs]

I would encourage you to aim for wine compatibility and ease of use rather than full linux port. If you want to create a linux native VFS that is fine but I think that trying to change the MO2 code to make it linux native is a waste as it would also make it very hard for you to keep your port up to date with our changes.

If on the other hand you want to concentrate on the VFS thing and modularize the way Mo2 does virtualization that could actually be something we could merge into the main code. Then linux users can use your VFS and normal users can use either USVFS or other solutions like KxVFS (Kortex Mod Manager VFS) or links or whatever new solutions we might want to add in the future.

This way everyone benefits and linux users could stay up to date with our changes.

Changing some of the main code that is windows specific, like the directory structure updater (conflict detection code in Mo2) is something that I would advise against as it was optimized to be as fast as possible by using low level windows directory iteration functions to be able to iterate hundred of thousands of files in less than a second for example. It's not jut a matter of making it platform independent and everyone is happy sadly.

Targeting wine is just what makes most sense I think, also considering that from what I understand a lot of games require Wine anyways.

[VelorumS]

Before going in I've written a general description (https://github.com/ChipmunkV/modorganizer).

tl;dr it's basically to try to keep the port up to date while not replacing Windows-specific code with a cross-platform one. Of course upstream the bugfixes to the existing cross-platform code.

[AnyOldName3]

For a fair bit of the Windows-specific code, it's got no need to stay Windows-specific. If, for example, there's an equivalent Qt call that can be made and we've got some conversion from QStrings to UTF-16 LPWSTR before we call a Win32 function and convert the result back to QString, obviously we can upstream the change as it makes things easier to read and probably more robust. The main thing to watch for is things that look equivalent, but aren't. For example, I've been caught out by using QSettings to write to an INI file the game reads with GetPrivateProfileString as they're subtly different (e.g. backslashes and newlines in stored strings).

[HiPhish]

Sorry for bumping the closed thread, but I want to voice my question in public where it can be seen by everyone. I want to understand how MO2 is tricking a game into loading its mods. I have had almost success with Overlay Filesystem, but not quite. Here is what I did:

• Install the GOG version of Oblivion
• Install 4GB patch and xOBSE manually
• Install Darnified UI through MO2 as a manual download

When I lauch Oblivion through MO2 the new UI works. If I launch Oblivion on its own the UI does not work. This is to be expected. Next I mounted a virtual file system:

• Lower layers:
  • C:\users\hiphish\AppData\Local\ModOrganizer\Oblivion\mods\DarNified UI
  • C:\GOGGames\Oblivion\Data
• Mount point C:\GOGGames\Oblivion\Data

I also tried swapping the two lower directories, same result. The UI works, but the little icon in the lower left corner of the menu is missing, and the background of the character menus is completely transparent. So obviously I have some of the data, but some stuff is still missing. I though the problem might be that I am mounting into a directory that is also one of the layers, but the same thing happens if I rename the original Data folder.

I launch the game by first cding into the game directory, then I call wine obse_loader.exe. My question is, what else is MO2 doing? I think we first need to understand and replicate by hand the effect of MO2 before we can think about porting it.

[JoaoVitorSCS]

Sorry for bumping the closed thread, but I want to voice my question in public where it can be seen by everyone. I want to understand how MO2 is tricking a game into loading its mods. I have had almost success with Overlay Filesystem, but not quite. Here is what I did:

* Install the GOG version of Oblivion

* Install 4GB patch and xOBSE manually

* Install Darnified UI through MO2 as a manual download

When I lauch Oblivion through MO2 the new UI works. If I launch Oblivion on its own the UI does not work. This is to be expected. Next I mounted a virtual file system:

* Lower layers:
  
  * `C:\users\hiphish\AppData\Local\ModOrganizer\Oblivion\mods\DarNified UI`
  * `C:\GOGGames\Oblivion\Data`

* Mount point `C:\GOGGames\Oblivion\Data`

I also tried swapping the two lower directories, same result. The UI works, but the little icon in the lower left corner of the menu is missing, and the background of the character menus is completely transparent. So obviously I have some of the data, but some stuff is still missing. I though the problem might be that I am mounting into a directory that is also one of the layers, but the same thing happens if I rename the original Data folder.

I launch the game by first cding into the game directory, then I call wine obse_loader.exe. My question is, what else is MO2 doing? I think we first need to understand and replicate by hand the effect of MO2 before we can think about porting it.

Hi man, can you check which archives the mod in question should overwrite? check if both are either uppercase or lowercase

[HiPhish]

Hi man, can you check which archives the mod in question should overwrite? check if both are either uppercase or lowercase

Yes, that was it! The mod has a directory called textures while the games's data has Textures. Fixing the mod gets the UI to load correctly. So MO2 "just" overlays directories. A few points of interest:

• Case sensitivity is a pain, it looks like the game picked the name that comes first in ASCII or Unicode (upper-case letter have a lower code point than lower-case)
• Mounting through mount requires root permissions, so that's a pain point
• We can use fuse-overlayfs to mount an overlay file system without root permissions, but then we cannot mount into one of our lower directories. This means the user would have to first rename the game'sData to _Data or whatever. I don't know if that is a bug in fuse-overlayfs or a limitation of FUSE in general (see Cannot use one of the lower directories as mount target containers/fuse-overlayfs#344).
• The mount is visible to the entire OS, but we could rectify that through mount namespaces. I have not yet tried it.

I think case sensitivity is the biggest problem. One solution would be to run over all file trees of the game and mods and lower-case the names. There is also ciopfs (case-insensitive on purpose file system) which implements a case-insensitive FS on top of an existing directory, but I have not tried it. Maybe the combination of this and the overlay file system could solve the issue, assuming that it actually merges the contents of textures and Textures instead of letting one win.

If all else fails we might have to write our own FUSE file system specifically for this use case. It is something I would rather avoid though.

[JoaoVitorSCS]

Actually the case sensitive issue comes from both system filesystems, windows using shitty NTFS and FAT32 and linux using EXT4, EXT4 is case sensitive so when you try to replace a file/folder it won't recognize if you're using it outside wine bottle by mounting it manually, but if you run it using mod organizer 2 it will work because it simulates NTFS behavior replacing it successfully, as you're using GOG try installing the game manually by the auto installer and then installing manually the mod organizer 2 and the same mod inside it, by my last experiment i got it working, but trying to setup with steam is a pain in the ass

[HiPhish]

MO2 works fine for me in Wine (aside from the UI and opening Nexus links, but that's a different story). I was intentionally trying to launch a modded game outside MO2 to figure out if I could replicate its mod organizer feature in order to better understand it. As an added bonus, if we can get the loading of mods working outside of MO2 we can use MO2 just as a mod unpacker and use native Linux tools for the rest. This would reduce the reliance on MO2.

[alexzk1]

Usvfs, the virtual file system Mo2 uses, hooks windows filesystem functions to change their behavior. It can't be "ported". Someone would have to create a VFS specifically designed for Linux and make it work for Mo2.

It was estimated that a working port of Mo2 would require multiple years of work of low level Linux knowledgeable developers. None of the active developers is interested Linux or provind support for it. We are happy to welcome new developers but if their intention is to make a Linux version we would warn them it's not work their time

That said a lot of people have managed to get Mo2 to work on Wine. For more information you can check the thread we kept open for it. We don't support it but if it works all the better.

#372

Hope you can work something out there

Actually, this idea itself is back-port from Linux: https://en.wikipedia.org/wiki/Filesystem_in_Userspace
So it's just built in.
Need Linux based gui part with all stuff + some small exe with basic functions to be launched by steam/wine and communicate to Linux part.
Right now UI part is barely usable in wine - lags, checkbox click takes 20s. Degrades - works slower and slower, I have to restart it each 3-4 mods.
Upd: fixed slowness - use Proton Experimental (8.xx today) despite all guides say to use 6th.

[AnyOldName3]

Fuse is just a library for implementing things like USVFS, and on its own replaces maybe a few percent of MO2's Windows-specific code at most. If it genuinely did everything, surely you could post a link to a Fuse-based MO2 alternative that you'd made instead of just a comment telling us to make one.

[alexzk1]

Patience please :) I just got this game/mo2. But well, as I updated there - with Proton Experimental it is OK as is. Need to update guides possibly which insist to use proton 6.

[Ruedii]

Looking at the libraries called it should* be directly portable to Linux with little effort. It would be nice.
This of course means either having the user manually enter the game directory locations or keeping a nice list of the places to look.

As of the overlay FS, Linux has a similar function already without need of userspace support. You only need mount permission in your user directory:
https://docs.kernel.org/filesystems/overlayfs.html

Making it run on Proton on the other hand may or may not be easy, depending on current blockers. Is there a log mechanism on the program. I see it is giving dumps but no logs. Should I just use Proton's internal logs instead?

*should and is are two different things. Murphy's law likes to be proven.

[Ruedii]

BTW, I always find the Arch Wiki provides better user level documentation.
https://wiki.archlinux.org/title/Overlay_filesystem

[AnyOldName3]

Some of what you're bringing up is discussed above. That which isn't can be found numerous times in our development Discord server. It's probably best if you track some of that down and take a look if you're interested in this, but it'll be pretty demoralising.