ProtonUp installing ProtonGE into wrong directory for steam

[ndlarsen]

Please fill out following when reporting a new bug:

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. Install from flathub
2. add proton GE to steam
3. launch Steam
4. go to aettings -> compatability
5. observe recently installed version missing from drop down list

Expected behavior
Installed version available via Steam
Desktop (please complete the following information):

• Platform: Are you using a Steam Deck or a PC/Laptop?
  PC
• System:

  $ lsb_release -a
  No LSB modules are available.
  Distributor ID:	Ubuntu
  Description:	Ubuntu 20.04.6 LTS
  Release:	20.04
  Codename:	focal
  

• Version:

  $ flatpak info net.davidotek.pupgui2 
  
  DavidoTek - Install Wine- and Proton-based compatibility tools
  
  ID: net.davidotek.pupgui2
  Ref: app/net.davidotek.pupgui2/x86_64/stable
  Arch: x86_64
  Branch: stable
  Version: 2.10.0
  License: GPL-3.0
  Origin: flathub
  Collection: org.flathub.Stable
  Installation: system
  Installed: 100,6 MB
  Runtime: org.kde.Platform/x86_64/6.7
  Sdk: org.kde.Sdk/x86_64/6.7
  
  Commit: 510d1791f4d18d9d80c5e68aed5bf4c9922ec5977a67651c2537072b924adefd
  Parent: 2cae287490b7e653ad4022f2f5ed8bd835ec8481e9dd0023f6061019915a0d2a
  Subject: python3-steam: Ignore dependencies (df0f1eea)
  Date: 2024-08-20 19:48:13 +0000
  

• How did you install ProtonUp-Qt?:
  Flatpak via Flathub

Additional context
ProtonUp installs the runtimes into ~/.steam/steam/compatibilitytools.d/ but on my system all other runtimes need to be installed into ~/.steam/root/compatibilitytools.d/ where ~/.steam/root is a symbolic link to ~/.steam, so effectively ~/.steam/compatibilitytools.d/. This is how is has always been without any tampering on my part.

Way back Steam was installed via the steam-installer package provided by Ubuntu:

$ apt-cache policy steam-installer 
steam-installer:
  Installed: 1.0.0.61-2ubuntu3
  Candidate: 1.0.0.61-2ubuntu3
  Version table:
 *** 1.0.0.61-2ubuntu3 500
        500 http://dk.archive.ubuntu.com/ubuntu focal/multiverse amd64 Packages
        500 http://dk.archive.ubuntu.com/ubuntu focal/multiverse i386 Packages
        100 /var/lib/dpkg/status

Terminal output

# Open the Terminal app and type one of the following commands and press Enter (The app is called "Konsole" on Steam Deck):

# If you installed ProtonUp-Qt via Flatpak (Discover on Steam Deck), type:
# flatpak run net.davidotek.pupgui2


$ flatpak run net.davidotek.pupgui2 
ProtonUp-Qt 2.10.0 by DavidoTek. Build Info: DavidoTek Flathub build.
Python 3.11.9 (main, Nov 10 2011, 15:00:00) [GCC 13.2.0], PySide 6.7.0
Platform: KDE Flatpak runtime 6.7 Linux-5.15.0-122-generic-x86_64-with-glibc2.38
Loading locale en / en_US
Loaded ctmod GE-Proton
Loaded ctmod Wine-GE
Loaded ctmod Boxtron
Loaded ctmod Kron4ek Wine-Builds Vanilla
Loaded ctmod Lutris-Wine
Loaded ctmod Luxtorpeda
Loaded ctmod Northstar Proton (Titanfall 2)
Loaded ctmod Proton Tkg
Loaded ctmod Proton Tkg (Wine Master)
Loaded ctmod Roberta
Loaded ctmod Steam-Play-None
Loaded ctmod SteamTinkerLaunch
Loaded ctmod SteamTinkerLaunch-git
Loaded ctmod vkd3d-lutris
Loaded ctmod vkd3d-proton
Loaded ctmod Wine Tkg (Valve Wine Bleeding Edge)
Loaded ctmod Wine Tkg (Wine Master)
Loaded ctmod DXVK
Loaded ctmod DXVK Async
Loaded ctmod DXVK (nightly)
Gamepad error: No gamepad found.

[sonic2kk]

Is ~/.steam/steam a valid Steam installation, i.e. does it have ~/.steam/steam/config/config.vdf and ~/.steam/steam/config/libraryfolders.vdf? If so, then ProtonUp-Qt is going to detect this as an active Steam installation.

Although looking at the code I would expect that it would also detect ~/.steam/root. But maybe since ~/.steam/root is a symlink and a symlink to to ~/.steam which we don't list in our valid installation paths, it is being ignored.

Can you check if the launchers dropdown contains any other detected Steam installations as well?

[ndlarsen]

I appreciate your reply.

Is ~/.steam/steam a valid Steam installation, i.e. does it have ~/.steam/steam/config/config.vdf and ~/.steam/steam/config/libraryfolders.vdf?

$ ls ~/.steam/steam/config/config.vdf ~/.steam/steam/config/libraryfolders.vdf
/home/ndlarsen/.steam/steam/config/config.vdf  /home/ndlarsen/.steam/steam/config/libraryfolders.vdf

So, yes and yes.

Contents of .steam/root

$ ls .steam/root
Backups               fontconfig    package       steamui-public  bin_steamdeps.py               registry.vdf                                  steamclient.dll  steam_subscriber_agreement.txt
bin                   friends       public        tenfoot         d3ddriverquery64.dxvk-cache    root                                          steamdeps.txt    steam.token
clientui              graphics      resource      ubuntu12_32     error.log                      sdk32                                         steam_msg.sh     ThirdPartyLegalNotices.css
compatibilitytools.d  legacycompat  siteserverui  ubuntu12_64     fossilize_engine_filters.json  sdk64                                         steam.pid        ThirdPartyLegalNotices.doc
config                linux32       steam         bin32           GameOverlayRenderer64.dll      standalone_installscript_progress_228980.vdf  steam.pipe       ThirdPartyLegalNotices.html
controller_base       linux64       steamui       bin64           installscriptevalutor_log.txt  steamclient64.dll                             steam.sh         update_hosts_cached.vdf

Contents of .steam/steam

ndlarsen@master:~$ ls .steam/steam
appcache  cached  compatibilitytools.d  config  depotcache  games  logs  music  steamapps  userdata  local.vdf  network_config.json  ssfn4617878472297528854

If so, then ProtonUp-Qt is going to detect this as an active Steam installation.

While I understand that this is the default behaviour and probably desired behaviour in most cases, it does not change the fact that using this path does not work for me, only placing them in ~/.steam/root/compatibilitytools.d/ does (see below).

So, trying something out to prove the point.

1. I have cleaned out all currently installed custom proton versions.

   $ ll .steam/steam/compatibilitytools.d/
   total 8
   drwxr-xr-x  2 ndlarsen ndlarsen 4096 Sep 25 16:25 ./
   drwxr-xr-x 12 ndlarsen ndlarsen 4096 Sep 25 16:09 ../
   

   $ ll .steam/root/compatibilitytools.d/
   total 8
   drwxrwxr-x  2 ndlarsen ndlarsen 4096 Sep 25 16:10 ./
   drwxr-xr-x 24 ndlarsen ndlarsen 4096 Sep 25 16:09 ../
   

   $ ll .steam/root
   lrwxrwxrwx 1 ndlarsen ndlarsen 21 Sep 25 15:57 .steam/root -> /home/ndlarsen/.steam/
   

   
2. installed ProtonGe 9.14 via ProtonUp-QT
   

   $ ls .steam/steam/compatibilitytools.d/
   GE-Proton9-14
   

3. uninstalling ProtonGe 9.14 via ProtonUp-QT
4. maunally installing ProtonGE 9.13 into ~/.steam/root/compatibilitytools.d/

 $ ls .steam/root/compatibilitytools.d/
 GE-Proton9-13

I am aware that the de facto location of compatibilitytools.d has been changed from Valve's side more than once (see below) so this is probably a leftover from earlier versions of the Steam client. If that is the case, one could ague (and I am) that ProtonUp-QT should consider supporting this as previous default locations clearly still has effect for some.

Known changes to the location of compatabilitytools.d:

1. `~/.local/share/Steam/compatibilitytools.d`
2. `~/.steam/root/compatibilitytools.d`
3. `~/.steam/steam/compatibilitytools.d` (currrent)

Although looking at the code I would expect that it would also detect ~/.steam/root. But maybe since ~/.steam/root is a symlink and a symlink to to ~/.steam which we don't list in our valid installation paths, it is being ignored.

Symlink or not should not matter. My installed versions of ProtonGE are symlinks themselves and they work fine for Steam.

Can you check if the launchers dropdown contains any other detected Steam installations as well?
It does not, for Steam, only ~/.steam/steam/compatibilitytools.d/ is detected. (Taking a screenshot of it is not an option as the dropdown is closed when a screenshot app is openend due to change of focus and the dropdown hogs the keyboard while expanded)

A side effect of this issue is also that setting a compatibility tool for a game installed by ProtonUp-QT via ProtonUp-QT causes the game to crash during startup as the installed compatibility tool does not exist from the Steam client's point of view.

Let me repeat that this is an somewhat old, that is pre ~/.steam/steam/compatibilitytools.d, installation of steam but others are bound to run into this as well.

[sonic2kk]

ProtonUp-Qt does not to my knowledge only choose one or choose any "default" install location. It should list all known valid Steam installations in the dropdown. I think you have entirely misunderstood me, I was trying to illustrate why ProtonUp-Qt was picking up this installation, I was not trying to say anything about "desired" picking of a Steam installation.

You should remove leftover Steam installations if the one you mentioned is indeed a leftover, as good practice so that tools can pick the correct one.

Symlink or not should not matter.

Again, I think you have misunderstood. What I meant by this is that ProtonUp-Qt will attempt to filter out duplicates. For example if it picks up three Steam installations, but one is a symlink, listing three Steam installations in the dropdown would be redundant. ProtonUp-Qt tries to only detect valid, unique Steam installations. That's where symlinks come in. I was not referring to GE-Proton symlinks or symlinks of other compatibility tools, but symlinks that Steam itself creates for its installations (i.e. on Arch-based distros, typically ~/.steam/root is a symlink to ~/.local/share/Steam).

I don't understand the crashing thing you're referring to, could you provide more details?

---

Also, sorry if I misunderstood your reply or missed it, but does ProtonUp-Qt list both Steam installations in the dropdown? It should be able to detect both of both contain the VDF files listed. If does not pick one or the other if I remember correctly, you should be able to see both

[ndlarsen]

@sonic2kk

Thank you for clarifying.

You should remove leftover Steam installations if the one you mentioned is indeed a leftover

I do not believe this is a leftover installation as such more that a consequence of how the steam installer has been packaged by Ubuntu years ago before Valve provided their own.

I don't understand the crashing thing you're referring to, could you provide more details?

If I install a version of e.g. ProtonGE via ProtonUp-QT and via ProtonUp-QT set the previously installed version of ProtonGE as default for a steam game, then the same steam game fails to launch until the changes are reverted.

Also, sorry if I misunderstood your reply or missed it, but does ProtonUp-Qt list both Steam installations in the dropdown?

No, only ~/.steam/steam/compatibilitytools.d which is not detected by Steam itself.

I have made an issue at https://github.com/ValveSoftware/steam-for-linux/issues, trying to determine which location is in fact the correct one and having asked for official documentation on the matter. Will post the response here if I get a usable one from Valve but what I gather from it so far is that there is an order of precedence across a list of folders and environment variables that should be considered.

[ndlarsen]

@sonic2kk So, I deleted the ~/.steam folder entirely, made sure that ~/.local/share/Steam did no exist as well as replaced the steam client packaged by Ubuntu with the Steam client provided by Valve attempting to ensure everything is as close to intended by Valve as possible.

$ ll ~/.steam
total 20
drwxrwxr-x   2 ndlarsen ndlarsen 4096 Sep 25 20:53 ./
drwxr-xr-x 110 ndlarsen ndlarsen 4096 Sep 25 20:55 ../
lrwxrwxrwx   1 ndlarsen ndlarsen   27 Sep 25 20:52 bin -> /home/ndlarsen/.steam/bin32/
lrwxrwxrwx   1 ndlarsen ndlarsen   45 Sep 25 20:52 bin32 -> /home/ndlarsen/.local/share/Steam/ubuntu12_32/
lrwxrwxrwx   1 ndlarsen ndlarsen   45 Sep 25 20:52 bin64 -> /home/ndlarsen/.local/share/Steam/ubuntu12_64/
-rwxrwxr-x   1 ndlarsen ndlarsen  583 Sep 25 20:53 registry.vdf*
lrwxrwxrwx   1 ndlarsen ndlarsen   33 Sep 25 20:52 root -> /home/ndlarsen/.local/share/Steam/
lrwxrwxrwx   1 ndlarsen ndlarsen   41 Sep 25 20:52 sdk32 -> /home/ndlarsen/.local/share/Steam/linux32/
lrwxrwxrwx   1 ndlarsen ndlarsen   41 Sep 25 20:52 sdk64 -> /home/ndlarsen/.local/share/Steam/linux64/
lrwxrwxrwx   1 ndlarsen ndlarsen   33 Sep 25 20:52 steam -> /home/ndlarsen/.local/share/Steam/
-rw-rw-r--   1 ndlarsen ndlarsen    6 Sep 25 20:52 steam.pid
prw-------   1 ndlarsen ndlarsen    0 Sep 25 19:37 steam.pipe|
-r--------   1 ndlarsen ndlarsen   16 Sep 25 20:52 steam.token

Now both ~/.steam/root and ~/.steam/steam are symbolic links to /home/ndlarsen/.local/share/Steam/ which means ~/.steam/root/compatibilitytools.d/ and ~/.steam/steam/compatibilitytools.d/ are the same folder.

ProtonUp-QT now detects ~/.local/share/Steam/compatibilitytools.d.

If I manually create a symbolic link in ~/.local/share/Steam/compatibilitytools.d (and thus ~/.steam/root/compatibilitytools.d and ~/.steam/steam/compatibilitytools.d) pointing towards the location of e.g. ProtonGE 9.13, then Steam detects it fine but ProtonUp-QT fails to. If I install ProtonGE 9.13 via ProtonUp-QT without removing the symbolic link, ProtonUp-QT fails because the link exists. If everything is clean and dandy, the installing via ProtonUp-QT works as expected. ProtonUp-QT clearly has an issue with symbolic links that might be worth looking into.

However, I do not think this explains why ProtonUp-QT failed to detect the "steam root" which Steam itself detected earlier but at this point I do not care about that part any more.

Now, I promised to convey some information from another thread about "officially supported folders" and the order of precedence, so here goes: ValveSoftware/steam-for-linux#6310 and based on that it seems that ~/.steam/root/compatibilitytools.d is officially endorsed.

Regards