Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does this work as an offline installer? #410

Closed
Darthagnon opened this issue Jun 19, 2024 · 6 comments
Closed

Does this work as an offline installer? #410

Darthagnon opened this issue Jun 19, 2024 · 6 comments
Labels
enhancement New feature or request

Comments

@Darthagnon
Copy link

Does ProtonUp-Qt work as an offline installer for Proton, Wine, etc.?

When I get a Steam Deck, I'm planning to install Windows on it purely because there are offline installers available for most software that will work without an internet connection to package managers' repos.

I wish Linux had the same philosophy, and was wondering whether this tool would install these dependencies without internet?

@Darthagnon Darthagnon added the enhancement New feature or request label Jun 19, 2024
@DavidoTek
Copy link
Owner

No, ProtonUp-Qt downloads all compatibility tools from GitHub.
It wouldn't be possible to include all possible tools because they would use up several hundreds of gigabytes.

You can download GE-Proton as an archive file without using ProtonUp-Qt from another computer and transfer it using a thumb drive in theory...

@sonic2kk
Copy link
Contributor

Much software on Linux can be installed offline if not installed via a package manager; for example, AppImages.

ProtonUp-Qt can be used offline via the AppImage, but as mentioned it cannot install tools offline because they are fetched from external sources. Much functionality does work offline though, like the Shortcut Editor, Games List, Batch Update. It can still help you manage manually installed tools.

Regardless of operating system, tools that are a frontend for installing software from the web will always require an Internet Connection. It was recently (a matter of hours ago, #412) proposed to add functionality to extract a given archive into a compatibility tool folder using ProtonUp-Qt. Should such a proposal be eventually implemented, it could aid in offline usage.

@Darthagnon
Copy link
Author

Darthagnon commented Jun 22, 2024

No, ProtonUp-Qt downloads all compatibility tools from GitHub. It wouldn't be possible to include all possible tools because they would use up several hundreds of gigabytes.

You can download GE-Proton as an archive file without using ProtonUp-Qt from another computer and transfer it using a thumb drive in theory...

GE-Proton is only 415mb? I'm not too sure where to manually extract it, but #412 would help a lot! (EDIT: Typo, I meant 412, not 415)

@sonic2kk
Copy link
Contributor

sonic2kk commented Jun 22, 2024

One version of GE-Proton is only 415mb. Multiply that by all the releases, even just the last few. Plus, there's no way for ProtonUp-Qt to get the latest version unless it packaged a version with each release, which is unreasonable.

On top of this, ProtonUp-Qt also allows you to install other Proton versions. Even just the ones with Advanced Mode disabled, Proton-tkg stands out as another large one. There's also Kron4ek Wine builds, which there's reason why you might not want to use the latest so it's impossible to know which one should be downloaded without packaging several versions.

Expanding beyond Steam there's also Wine-GE which is of similar size, and Wine-tkg. Not to mention the variations thereof (builds from Wine Master can be up to 1gb!) and those are downloaded from CI, so there are several builds.

Even just packaging a few releases of each tool for each launcher we're already into tens of gigs. And the ones we pre-package would have to be updated with each release, some of which would get outdated very quickly. And ProtonUp-Qt would then have to update very often just to package newer archives to extract. This means the releasing strategy would also have to change, as there may be in-progress features in master that are pending follow-up PRs to finalize.

Not to mention here either the logistics of adding the releases into the archive. Short of manually placing the archives into ProtonUp-Qt and manually uploading a release, these would need to be downloaded and injected on CI, which would take a long time. ProtonUp-Qt would also have to be updated so Ctmods knew where to find the built-in ones. Users then also have to trust the archives ProtonUp-Qt has built in as blobs, because they aren't available on the repo, and we probably won't want to build each Proton version ourselves on CI so we can't add them as subprojects or something. True users also have to trust when we download the archives but the source for how we do that is available and it's much easier to vet Python code that downloads an archive than it is to vet blobs in releases.

I also somewhat challenge the "only" 415mb. The ProtonUp-Qt AppImage is 90mb, approximately a quarter of that size. On the flip side, the Flatpak if you don't have the KDE Runtime is quite large. We don't want to balloon the package sizes. Even just including the latest GE-Proton (which wouldn't make sense, there's no reason for GE-Proton to get special treatment other than bias) that would, at best, double the filesize. And these tools are already compressed as much as they probably can be so further compression is unlikely to make a significant enough difference that packaging the tools becomes feasible.

It is overall in my opinion just not feasible to include the tools in ProtonUp-Qt. It would be cumbersome to maintain, drastically increase file size, and he harder to vet from a security perspective.

I'm not too sure where to manually extract it

You can manually extract it to the folder that ProtonUp-Qt shows on the Main Menu, beside the launcher name. If you cannot see the full path on the window, you can click on the dropdown to view the full path. This can be done fully offline if you use the AppImage for example.

image

If you have any tools installed already, you can select that tool on the Main Menu, click "Show info", and the "Install directory" label will display the path. You can then copy and paste this path into your file manager. However having a tool installed would require you to have either already installed a tool to this location

image

It would also be possible to try to manually find the path. It depends on your distribution and how you installed Steam, but the main paths are as follows:

  • ~/.local/share/Steam/compatibilitytools.d/ for Arch Linux (and most of its derivatives like Garuda, Manjaro, and endeavourOS) vanilla SteamOS.
  • ~/.steam/root/compatibilitytools.d/ usually exists for most installations but may not for others.
  • ~/.steam/debian-install/compatibilitytools.d/ for some Debian-based distro installations (notably not all, i.e. some Ubuntu/Debian installations seem to use this and some do not).
  • ~/.steam/steam/compatibilitytools.d is occasionally the only available path.
  • ~/.var/app/com.valvesoftware.Steam/data/Steam/compatibilitytools.d for the unofficial Steam Flatpak.
  • ~/snap/steam/common/.steam/root/compatibilitytools.d/ for the unofficial Steam Snap.

These paths are noted in constants.py but admittedly it is difficult to infer the paths if you do not know Python. Still, if you're interested, that's how ProtonUp-Qt determines the Steam paths.

This file also contains the paths for other launchers, although there is some Ctmod-specific logic to extract Lutris ctmods into, for example, the /runtimes directory instead of the /runners/wine directory.

@Darthagnon
Copy link
Author

Darthagnon commented Jul 21, 2024

One version of GE-Proton is only 415mb. Multiply that by all the releases, even just the last few. Plus, there's no way for ProtonUp-Qt to get the latest version unless it packaged a version with each release, which is unreasonable.
[...] without packaging several versions.
Even just packaging a few releases of each tool for each launcher [...]

Please do not include any dependency package releases with ProtonUp-Qt; I didn't mean to suggest such a ridiculous idea 😅. Keep the current download function, but add a button for the user to select their own ZIPs of Proton, and it can install from ZIP. That's usually how such tools work on Windows (closest equivalent is probably ModOrganizer2 for Skyrim etc.)

It is overall in my opinion just not feasible to include the tools in ProtonUp-Qt. It would be cumbersome to maintain, drastically increase file size, and he harder to vet from a security perspective.

I fully agree with this.


I've noticed that in the development of Linux tools, it is a common assumption that we need all versions of a tool/dependency available all the time (only possible by maintaining lists of URLs in package managers; any URL is a 404 waiting to happen).

I come from Windows, where we never archive all versions of a tool/dependency, only the latest, or maybe a couple recent or critical ones, and save them for later. Dependencies should always be separate packages, but should always work without internet if long-term usability is desirable. In my day job, involving industrial Linux controllers running Debian 9 or custom packaged Raspberry Pi on Jessie, or with jailbroken iOS machines I use, I've run into lots of problems with the deprecation or archival of package repositories, so I've taken to backing up DEBs and ZIPs (a pain, because Linux does not make offline installation default or easy). Whereas with Windows XP machines we used to talk to old equipment, no such issues, because on Windows the default assumption is that the installer must work offline. Or, to use the ModOrganizer2 example, I can still use Skyrim mods from 10 years ago that the mod author deleted, because I can install from ZIP.

@sonic2kk
Copy link
Contributor

sonic2kk commented Jul 21, 2024

Package managers are intended to fetch packages and dependencies from repositories. That is by design. What you probably want is something that has all dependencies bundled together so that they don't have to be installed, something like an AppImage. Tools are not installed on Linux the same way they are on Windows most of the time. You don't have to put programs into a specific installation folder, there are no installation wizards, portable programs generally don't need to be installed via package manager. And for what it's worth, package managers by default are expected to connect to the internet and download dependencies (Scoop on Windows iirc, Homebrew on macOS).

With package managers you can even point them to your own repos, at least you can do this with Pacman on Arch and I'm almost certain you can do it with Scoop. I'm not sure about apt on Debian-based distributions (since you mentioned Debian) or other package managers. But you can absolutely keep all your dependency archives and tell your package manager to fetch from those. The reason they connect online by default is because they're configured to fetch from that distribution's server. With Pacman you can install from a PKGBUILD file which can be configured to handle dependencies separately. But of course package managers will default to fetching online, they do this on Windows too, it just happens that on Windows the default mentality is not to install from online but to use frontend installers that move files from the downloaded folder to an installation folder higher up the filesystem. You can do this on Linux too without any extra effort, but you don't even need to move them. You can just use binaries that are statically linked, or AppImages, and so on.

If you want to "install" tools offline you don't need a package manager, you want something like an AppImage with AppImageLauncher, or you want something designed to be portable (i.e. the way the Godot game engine is distributed). Usually portable binaries if I understand correctly will use static linking so they include the dependencies they need without relying on the system packages, or they'll expect to be run inside of a container (Proton, for example, relies on a container that Steam itself will download and keep up to date called the Steam Linux Runtime, and GE-Proton when used outside of Steam is moving to a similar solution that those launchers keep up-to-date called UMU).

Also, you can still install your compatibility tools offline if you have the ZIPs. ProtonUp-Qt is a good way to do it if you want a shortcut, but the default is still extracting into the compatibility tool / runtime folder. No one is stopping you from doing this or making it difficult. The comparison to ModOrganizer 2 doesn't work here, because ModOrganizer 2 creates a virtual filesystem and other shenanigans in most cases including Skyrim.

Much Linux software does work offline and can do things offline. It happens that in your examples you're pointing to software designed with a network connection in mind. This would be like complaining that Steam requires an Internet Connection to install games, these are network-reliant tools that fetch things from the Internet. ProtonUp-Qt doesn't do anything that, provided you have the archives, you couldn't also do manually very quickly. The main benefit of ProtonUp-Qt imo (or similar tools) is that it can make downloading the archives easier, presenting the list of tools and versions. It's not like a package manager in the example you gave, which does a lot more legwork than ProtonUp-Qt.

If you already have the archives it is probably much faster to just bulk extract them into the given launcher's folder. But if you don't have them and want to install a bunch of tools at once, that's where the value of ProtonUp-Qt comes in. When you already have the archives though there is no reason you can't extract the tools yourself. This is how it worked before ProtonUp-Qt, and it is now many people still do it.

Linux is not Windows, Windows users do have to adjust to the Linux way for sure (where package managers are standard but not the only way, and if you want the experience you describe, you need not use a package manager), but it is not as you have categorized.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants