Vcpkg export is trying to export packages from the wrong triplet #24033

mmahnic · 2022-04-08T06:45:13Z

Describe the bug

When exporting packages for the triplet x64-windows-v143, vcpkg wants to include also packages from the triplet x64-windows. Export fails, although the packages it requires are in fact present in x64-windows. These are the "native" packages that are needed to build the packages in the "target" triplet (x64-windows-v143). Vcpkg suggests that these packages should be installed also in x64-windows-v143:

Additional packages (*) need to be exported to complete this operation.
There are packages that have not been built.
To build them, run:
vcpkg install boost-build:x64-windows-v143 boost-modular-build-helper:x64-windows-v143 pkgconf:x64-windows-v143 vcpkg-pkgconfig-get-modules:x64-windows-v143 vcpkg-tool-meson:x64-windows-v143

But at least vcpkg-tool-meson can not be installed:

Error: vcpkg-tool-meson[core] is only supported on 'native'

We use different compilers so we define additional triplets that include the platform toolset. This is the content of the triplet file for x64-windows-v143:

set(VCPKG_TARGET_ARCHITECTURE x64)
set(VCPKG_CRT_LINKAGE dynamic)
set(VCPKG_LIBRARY_LINKAGE dynamic)
set(VCPKG_PLATFORM_TOOLSET v143)

Environment

OS: Windows
Compiler: VC2022, VC2019

To Reproduce
Steps to reproduce the behavior:

./vcpkg install --triplet x64-windows-v143 boost
Create the package list:

   triplet=x64-windows-v143
   ./vcpkg.exe list --triplet $triplet \
      | gawk "{ print \$1 }" | sed -e "s/:.*$//" | sort | uniq \
      | sed -e "/\[.*\]/d" \
      > $triplet.list

export

      ./vcpkg export --triplet $triplet \
         --nuget --output=exported --nuget-version=123 \
         @$triplet.list \

Expected behavior
A nuget package "exported" is created.

The text was updated successfully, but these errors were encountered:

JackBoosY · 2022-04-08T07:07:08Z

cc @strega-nil-ms @Neumann-A

The tool port in the dependency should not be exported, or there is some problem here.

Neumann-A · 2022-04-08T07:56:04Z

I feel like a simple --host-triplet=$triplet is missing here. No reason to CC me for that.

mmahnic · 2022-04-08T07:59:36Z

I feel like a simple --host-triplet=$triplet is missing here. No reason to CC me for that.

Adding --host-triplet=x64-windows or --host-triplet=$triplet has no effect.

cenit · 2022-04-11T04:18:30Z

it should have been --host-triplet=x64-windows-v143 ;)

mmahnic · 2022-04-11T11:07:23Z

I have isolated the problem which is not with the export command but with the list command. The command ./vcpkg.exe list --triplet $triplet does not list only packages for the target triplet (x64-windows-v143) but also packages for the host triplet (x64-windows). Therefore the list of packages to export must be filtered:

 triplet=x64-windows-v143
   ./vcpkg.exe list --triplet $triplet \
      | grep $triplet \ 
      ...

Another thing that seems strange to me is that the ./vcpkg export --triplet $triplet also exports packages for the host triplet, which in many cases does not make sense. If we export x86-windows-v143 which we will use on a 32-bit system, the exported x64-windows packages can not be used on the target system.

I am closing this issue since filtering the output from list solved the problem, but I feel that the list and export commands should only work on selected target triplet and ignore anything from the host triplet, unless explicitly requested.

JackBoosY · 2022-04-12T01:19:16Z

Fine, I think this should be document.

cstamatopoulos · 2022-04-13T18:19:36Z

I don't see how this is not a bug. Even specifying the exact triplets brings in the default.

vcpkg.exe export --triplet x86-windows-static-md protobuf:x86-windows-static-md grpc:x86-windows-static-md --nuget --dry-run

The following packages are already built and will be exported:
  * abseil:x86-windows-static-md
  * abseil:x64-windows
  * c-ares:x86-windows-static-md
  * c-ares:x64-windows
  * grpc:x64-windows
    grpc:x86-windows-static-md
  * openssl:x64-windows
  * openssl:x86-windows-static-md
  * protobuf:x64-windows
    protobuf:x86-windows-static-md
  * re2:x64-windows
  * re2:x86-windows-static-md
  * upb:x64-windows
  * upb:x86-windows-static-md
  * vcpkg-cmake:x64-windows
  * vcpkg-cmake-config:x64-windows
  * zlib:x64-windows
  * zlib:x86-windows-static-md

quasarea · 2022-05-27T11:39:55Z

@mmahnic could you post the solution for filtering here please?
doing i.e:

.\vcpkg export --nuget --triplet=x86-windows --host-triplet=x86-windows grpc --nuget-id=grpc-x86

makes

The following packages are already built and will be exported:
  * abseil:x86-windows
  * abseil:x64-windows
  * c-ares:x86-windows
  * c-ares:x64-windows
  * grpc:x64-windows
    grpc:x86-windows
  * openssl:x64-windows
  * openssl:x86-windows
  * protobuf:x64-windows
  * protobuf:x86-windows
  * re2:x64-windows
  * re2:x86-windows
  * upb:x64-windows
  * upb:x86-windows
  * vcpkg-cmake:x64-windows
  * vcpkg-cmake-config:x64-windows
  * zlib:x64-windows
  * zlib:x86-windows
Additional packages (*) need to be exported to complete this operation.

and I don't see how I can influence the internal list command at this point. What exactly you expect users do to do to solve it?

mmahnic · 2022-05-28T09:52:24Z

I use this BASH script to export packages:

   triplet="x86-windows-v142"
   ./vcpkg.exe list --triplet $triplet --host-triplet=x64-windows \
      | grep $triplet \
      | gawk "{ print \$1 }" | sed -e "s/:.*$//" | sort | uniq \
      | sed -e "/\[.*\]/d" \
      > xdata/$triplet.list

   ./vcpkg export --triplet $triplet --host-triplet=x64-windows \
         --nuget --output=xdata/$nameprefix --nuget-version=$version \
         @xdata/$triplet.list \
      > xdata/$nupkgname.log

Set the variables nameprefix, version and nupkgname as you wish.

quasarea · 2022-05-30T09:48:39Z

@mmahnic thanks for this, I was not aware that I can feed triplet.list during export somehow, so it helps to sort it out.
Still, I would disagree with making the issue resolved as the current behavior is quite unexpected.

In my case it does not work as x64 packages are still pulled in:

invy · 2023-10-06T16:39:46Z

Why is this closed?
The filtering won't work if host and target triplets are equals.
Also why?
Why do I need to export vcpkg-*-helpers?
The problem lies withing vcpkg-tool, there is no way (as of now) to distinguish in the export plan between host and target packages and filter them out.
Please reopen or add the issue to vcpkg-tool.

Idealy I would like to build packages for different targets and export them maybe separately. So if I need later to cross-compile somewhere i.e. on x64-linux I would install host packages for x64-linux and target-packages for mingw64 so I'd be able to use tools in my build process.

sharun · 2024-10-29T19:36:16Z

We are also facing this issue where when the target is Android OS but the host is MAC_OS(where we are building it), then export tries to export the ones which are needed for the MAC_OS(host) and fails. Can we take a relook at this ?

This is amplified and hence this issue too.
microsoft/vcpkg-tool#1344

invy · 2024-10-29T21:14:36Z

I ended up writing my own script which exports packages how I need them. (I export in deb format).
Frankly speaking, exporter doesn't belong in the tool. Vcpkg should provide Api (and language bindings for other languages) for people to extend it. Like apt does this...

Vcpkg stores installed packages either in a 'status' file as empty line separated blocks or in separate files (manifest mode).
File lists are in info subdirectory.
Additionally you can parse contents of package manifests to find host-only dependencies.

JonLiu1993 assigned JackBoosY Apr 8, 2022

JackBoosY added the category:vcpkg-bug The issue is with the vcpkg system (including helper scripts in `scripts/cmake/`) label Apr 8, 2022

mmahnic closed this as completed Apr 11, 2022

JackBoosY added category:question This issue is a question and removed category:vcpkg-bug The issue is with the vcpkg system (including helper scripts in `scripts/cmake/`) labels Apr 12, 2022

sharun mentioned this issue Oct 30, 2024

Can't export prefabs: Your vcpkg 'installed' tree is corrupted. #35538

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Vcpkg export is trying to export packages from the wrong triplet #24033

Vcpkg export is trying to export packages from the wrong triplet #24033

mmahnic commented Apr 8, 2022 •

edited

Loading

JackBoosY commented Apr 8, 2022

Neumann-A commented Apr 8, 2022

mmahnic commented Apr 8, 2022 •

edited

Loading

cenit commented Apr 11, 2022

mmahnic commented Apr 11, 2022

JackBoosY commented Apr 12, 2022

cstamatopoulos commented Apr 13, 2022 •

edited

Loading

quasarea commented May 27, 2022

mmahnic commented May 28, 2022 •

edited

Loading

quasarea commented May 30, 2022 •

edited

Loading

invy commented Oct 6, 2023 •

edited

Loading

sharun commented Oct 29, 2024

invy commented Oct 29, 2024 •

edited

Loading

Vcpkg export is trying to export packages from the wrong triplet #24033

Vcpkg export is trying to export packages from the wrong triplet #24033

Comments

mmahnic commented Apr 8, 2022 • edited Loading

JackBoosY commented Apr 8, 2022

Neumann-A commented Apr 8, 2022

mmahnic commented Apr 8, 2022 • edited Loading

cenit commented Apr 11, 2022

mmahnic commented Apr 11, 2022

JackBoosY commented Apr 12, 2022

cstamatopoulos commented Apr 13, 2022 • edited Loading

quasarea commented May 27, 2022

mmahnic commented May 28, 2022 • edited Loading

quasarea commented May 30, 2022 • edited Loading

invy commented Oct 6, 2023 • edited Loading

sharun commented Oct 29, 2024

invy commented Oct 29, 2024 • edited Loading

mmahnic commented Apr 8, 2022 •

edited

Loading

mmahnic commented Apr 8, 2022 •

edited

Loading

cstamatopoulos commented Apr 13, 2022 •

edited

Loading

mmahnic commented May 28, 2022 •

edited

Loading

quasarea commented May 30, 2022 •

edited

Loading

invy commented Oct 6, 2023 •

edited

Loading

invy commented Oct 29, 2024 •

edited

Loading