Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merging vcpkg created sboms #377

Closed
KUGA2 opened this issue Jun 27, 2024 · 5 comments
Closed

Merging vcpkg created sboms #377

KUGA2 opened this issue Jun 27, 2024 · 5 comments

Comments

@KUGA2
Copy link

KUGA2 commented Jun 27, 2024

Has anyone tried this?

vcpkg generates a SPDX file containing the SBOM information for each package that is installed. The files are located in <installed_dir>//share//vcpkg.spdx.json.
https://learn.microsoft.com/en-us/vcpkg/reference/software-bill-of-materials

I was hoping to use this to merge them. Sadly, it does not work.

Reproduction:

  1. Build a vcpkg (manifest mode) project
    (i uploaded the jsons here for reproduction: inputs.zip )
  2. Run cyclonedx-cli merge:
$ docker run --rm -v <path>:<path> -v /tmp:/out cyclonedx/cyclonedx-cli merge --input-files $(ls <path>/vcpkg_installed/x64-linux-gcc/share/*/*.spdx.json) --output-file /out/sbom_all.json --hierarchical --name test_merge --version=1.0.0
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/abseil/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/argagg/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/c-ares/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/cppunit/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/czmq/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/docopt/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/grpc/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/gtest/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/libiconv/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/liblzma/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/libxml2/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/nanopb/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/openssl/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/protobuf/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/re2/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/upb/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/vcpkg-cmake-config/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/vcpkg-cmake-get-vars/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/vcpkg-cmake/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/zeromq/vcpkg.spdx.json
Processing input file <path>/vcpkg_installed/x64-linux-gcc/share/zlib/vcpkg.spdx.json
Unhandled exception: CycloneDX.Utils.Exceptions.MissingMetadataComponentException: Required metadata (top level) component is missing from BOM.
   at CycloneDX.Utils.CycloneDXUtils.HierarchicalMerge(IEnumerable`1 boms, Component bomSubject)
   at CycloneDX.Cli.Commands.MergeCommand.Merge(MergeCommandOptions options)
   at System.CommandLine.Invocation.CommandHandler.GetExitCodeAsync(Object value, InvocationContext context)
   at System.CommandLine.Invocation.ModelBindingCommandHandler.InvokeAsync(InvocationContext context)
   at System.CommandLine.Invocation.InvocationPipeline.<>c__DisplayClass4_0.<<BuildInvocationChain>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass23_0.<<UseParseErrorReporting>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass16_0.<<UseHelp>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass27_0.<<UseVersionOption>b__1>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass25_0.<<UseTypoCorrections>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<UseSuggestDirective>b__24_0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass22_0.<<UseParseDirective>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass11_0.<<UseDebugDirective>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<RegisterWithDotnetSuggest>b__10_0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass14_0.<<UseExceptionHandler>b__0>d.MoveNext()

(without hierarchical, it works but the result file is basically empty)
@mtsfoni
Copy link
Contributor

mtsfoni commented Jun 27, 2024

I'm not sure if you can merge spdx files directly, might need to transform them to CycloneDx first.

Are there maybe spdx merge tools, that would offer a more native solution?

@KUGA2
Copy link
Author

KUGA2 commented Jun 27, 2024

Oh, I see. Transform via cyclonedx convert, then merge. I will try.

@KUGA2
Copy link
Author

KUGA2 commented Jun 28, 2024

I did a similar a test with converted SBOMs but the result is the same (I guess, it converted implicitly):

 $ docker run --rm -v $(pwd):$(pwd) cyclonedx/cyclonedx-cli convert --input-file <vcpkg_installed>/libxml2/vcpkg.spdx.json --output-file <vcpkg_installed>/libxml2/vcpkg.cyclone.json --output-format json
 $ docker run --rm -v $(pwd):$(pwd) cyclonedx/cyclonedx-cli convert --input-file <vcpkg_installed>/nanopb/vcpkg.spdx.json  --output-file <vcpkg_installed>/nanopb/vcpkg.cyclone.json --output-format json
 $ docker run --rm -v $(pwd):$(pwd) cyclonedx/cyclonedx-cli merge --input-files $(ls <vcpkg_installed>/*/*.cyclone.json) --output-file $(pwd)/outputs/sbom_all.json --hierarchical --name test_merge --version=1.0.0  
Processing input file <vcpkg_installed>/libxml2/vcpkg.cyclone.json
    Contains 8 components
Processing input file <vcpkg_installed>/nanopb/vcpkg.cyclone.json
    Contains 6 components
Unhandled exception: CycloneDX.Utils.Exceptions.MissingMetadataComponentException: Required metadata (top level) component is missing from BOM.
   at CycloneDX.Utils.CycloneDXUtils.HierarchicalMerge(IEnumerable`1 boms, Component bomSubject)
   at CycloneDX.Cli.Commands.MergeCommand.Merge(MergeCommandOptions options)
   at System.CommandLine.Invocation.CommandHandler.GetExitCodeAsync(Object value, InvocationContext context)
   at System.CommandLine.Invocation.ModelBindingCommandHandler.InvokeAsync(InvocationContext context)
   at System.CommandLine.Invocation.InvocationPipeline.<>c__DisplayClass4_0.<<BuildInvocationChain>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass23_0.<<UseParseErrorReporting>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass16_0.<<UseHelp>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass27_0.<<UseVersionOption>b__1>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass25_0.<<UseTypoCorrections>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<UseSuggestDirective>b__24_0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass22_0.<<UseParseDirective>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass11_0.<<UseDebugDirective>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<RegisterWithDotnetSuggest>b__10_0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass14_0.<<UseExceptionHandler>b__0>d.MoveNext()

@andreas-hilti
Copy link
Contributor

andreas-hilti commented Jul 7, 2024 •
edited
Loading

@KUGA2 I think you get the best result if you convert explicitly and then do a flat merge (i.e. without --hierarchical).

Merging does not support the spdx format; instead it tries to read it as a cyclonedx json, which is the reason why the result of the flat merge above was basically empty.

For a hierarchical merge, you'd need have the metadata component available in the BOMs to be merged, which you don't have.

@KUGA2
Copy link
Author

KUGA2 commented Jul 16, 2024

This works:
sbom_all.json

But my original use case, importing into Black Duck, still shows no component. Closing this here.

@KUGA2 KUGA2 closed this as completed Jul 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@KUGA2 @andreas-hilti @mtsfoni