_VCPKG_INSTALLED_DIR should be a public parameter in vcpkg.cmake

[zomgrolf]

Hi,

Recently, I've been looking at manifests and versioning in vcpkg -- it does look quite promising. Our use case is slightly different from what vcpkg seems to be targeting at the moment, since we tend to build most of our top-level dependencies ourselves (usually applying some custom patches and other tweaks) and check them into our SCM, so some of the recent additions to vcpkg (binary caching and registries) seem less relevant to us -- I would greatly appreciate the ability to export installed libraries from a project that uses manifests and versioning, since we'd like to use vcpkg to handle building the dependencies of our top-level dependencies as it makes the entire process much easier!

This is about something slightly different, however -- the vcpkg CMake toolchain file that very neatly (I have to admit) integrates package management into CMake-based projects. At the moment, it would appear, when used in the manifest mode, the vcpkg toolchain file will by default install the libraries into the vcpkg_installed sub-directory of the build directory. That default seems to be controlled by the _VCPKG_INSTALLED_DIR CMake variable in the toolchain file. I take that the leading underscore is meant to indicate that this variable is an implementation detail and users should not touch it. I understand the desire to keep things simple, but I would argue that the ability to customize the location where the libraries are installed can be useful.

As an example, consider one of the libraries available in vcpkg: OpenImageIO. This is one of the libraries that we build ourselves, instead of installing it via vcpkg, since we need a custom build. As a part of the configuration process, it defines a target for its utility library (OpenImageIO_Util) and adds public headers from its dependencies (OpenEXR) as a part of its public interface. CMake spots that and produces the following error message (one of many):

  Target "OpenImageIO_Util" INTERFACE_INCLUDE_DIRECTORIES property contains
  path:

    "[...]/oiio/build/vcpkg_installed/x64-windows/include/OpenEXR"

  which is prefixed in the build directory.

Moving the vcpkg_installed directory out of the build dir by setting _VCPKG_INSTALLED_DIR to a directory outside the build directory solves the issue.

So I would appreciate the ability to customize the location of the vcpkg_installed library, when using the CMake toolchain file for vcpkg in manifest mode. That, or it shouldn't be put in the build directory by default, but I'm assuming you had good reasons to go with that choice.

[dg0yt]

we tend to build most of our top-level dependencies ourselves (usually applying some custom patches and other tweaks) and check them into our SCM, so some of the recent additions to vcpkg (binary caching and registries) seem less relevant to us

As far as I understand, registries is just for your case. There is a default registry (providing default ports), and there are additional registries (providing explicitly selected ports). You have to decide which registry is the default, vcpkg or your custom registry. Example vcpkg-configuration.json for the latter:

{
  "default-registry": {
    "kind": "filesystem",
    "path": "/your/custom/repository"
  },
  "registries": [
    {
      "kind": "git",
      "repository": "https://github.com/microsoft/vcpkg",
      "packages": [ "vcpkg-cmake", "vcpkg-cmake-config" ]
    }
  ]
}

[zomgrolf]

It doesn't really matter. We already have a working solution, have had it for many years in fact. It supports closed and open source products, multiple versions, multiple hardware architectures, OSes, toolchains and ABI settings. It's tailored to our needs and over the years it has been tuned to support key workflows and to integrate well with our build systems and CI/CD pipelines. We have established procedures for consuming dependencies, so the entire process is pretty much painless. But, of course, it's not compatible with vcpkg. And nobody is going to create more work for ourselves converting to vcpkg for what is a (relatively) minor benefit to the wider team.

I have nothing against this feature, I do see it as a welcome addition (I do love the composability aspect!), that many people will surely find useful. Perhaps even myself in a different setting.

[dg0yt]

I think that the benefits come with new team members, new components or new toolchains. At least this is why I want to switch from a well-working custom "Superbuild" (with CI, controlled sources, multi platform) to a more widely used package manager.

[dg0yt]

Moving the vcpkg_installed directory out of the build dir by setting _VCPKG_INSTALLED_DIR to a directory outside the build directory solves the issue.

There is an experimental parameter which changes the install root for all ports:

$ ./vcpkg --help
...
  --x-install-root=<path>         (Experimental) Specify the install root directory
...

To use this with build system integration in manifest mode, you will proably need to adjust VCPKG_INSTALL_OPTIONS or VcpkgAdditionalInstallOptions.

[zomgrolf]

Sadly, that doesn't really work, because of this bit in vcpkg.cmake:

  execute_process(
      COMMAND "${Z_VCPKG_EXECUTABLE}" install
          --triplet "${VCPKG_TARGET_TRIPLET}"
          --vcpkg-root "${Z_VCPKG_ROOT_DIR}"
          "--x-wait-for-lock"
          "--x-manifest-root=${VCPKG_MANIFEST_DIR}"
          "--x-install-root=${_VCPKG_INSTALLED_DIR}"
          "${Z_VCPKG_FEATURE_FLAGS}"
          ${Z_VCPKG_ADDITIONAL_MANIFEST_PARAMS}
          ${VCPKG_INSTALL_OPTIONS}
      OUTPUT_VARIABLE Z_VCPKG_MANIFEST_INSTALL_LOGTEXT
      ERROR_VARIABLE Z_VCPKG_MANIFEST_INSTALL_LOGTEXT
      RESULT_VARIABLE Z_VCPKG_MANIFEST_INSTALL_RESULT
      ${Z_VCPKG_MANIFEST_INSTALL_ECHO_PARAMS}
  )

If I try to pass --x-install-root=... in VCPKG_INSTALL_OPTIONS, vcpkg complains about --x-install-root being specified multiple times.

[zomgrolf]

It was actually the first thing I tried, before overriding _VCPKG_INSTALLED_DIR

[dg0yt]

If the issue is just the duplicate parameter in the context of VCPKG_INSTALL_OPTIONS, it should be fairly easy and safe to remove the undesired parameter in newer versions of CMake:

list(FILTER VCPKG_INSTALL_OPTIONS EXCLUDE REGEX "^--install-root=")

[zomgrolf]

The issue is that vcpkg.cmake always passes in the --x-install-root parameter when invoking vcpkg. If I try to use VCPKG_INSTALL_OPTIONS to override that default root with a custom location by setting VCPKG_INSTALL_OPTIONS to --x-install-root=my_location, then vcpkg reports an error about the install root being set multiple times. Which means, I can't use VCPKG_INSTALL_OPTIONS, I have to set _VCPKG_INSTALLED_DIR directly.

[dg0yt]

CMake spots that and produces the following error message (one of many)

Last not least, there are use cases which are not expected by upstream projects. Than you will have to patch their buildsystem.

[zomgrolf]

This much is obvious. The point of this exercise, however, is to reduce the amount of work one has to do, not increase it. In particular, I want to reduce the number of patching we have to do, because all such changes have to go through compliance checks and legal, and that's often a major pain. In this case, having to patch their build system to make it work with vcpkg means more work than not using vcpkg in the first place. Not to mention, that altering the way exported targets are defined, affects consumers of this library downstream, which makes vcpkg a really hard sell.

[dg0yt]

Understood. OTOH, this advice was meant for genuine bugs in upstream projects.