Duplicate components

[mmarseu]

When creating an SBOM for a large enough package, there are inevitably cases where a single dependency is included more than once. I guess that makes sense as long as you're not using the --flatten-components option.

When using that option, however, I wouldn't expect those duplicates to appear. They each only differ in their bom-ref.
Is that behavior intentional?

Edit: Sorry, I was a little pressed for time when I hit send. Let me give an example.

Assume the following dependency tree:

myPackage
|
|- [email protected]
|  | - [email protected]
|
|- [email protected]
   |- [email protected]

Expected output

All output shortened for clarity.

Without --flatten-components:

{
  "components": [
    {
      "purl": "pkg:npm/[email protected]",
      "bom-ref": "[email protected]",
      "components": [
        {
          "purl": "pkg:npm/[email protected]",
          "bom-ref": "[email protected]|[email protected]"
        }
      ]
    },
    {
      "purl": "pkg:npm/[email protected]",
      "bom-ref": "[email protected]",
      "components": [
        {
          "purl": "pkg:npm/[email protected]",
          "bom-ref": "[email protected]|[email protected]"
        }
      ]
    }
  ]
}

With --flatten-components:

{
  "components": [
    {
      "purl": "pkg:npm/[email protected]",
      "bom-ref": "[email protected]"
    },
    {
      "purl": "pkg:npm/[email protected]",
      "bom-ref": "[email protected]"
    },
    {
      "purl": "pkg:npm/[email protected]",
      "bom-ref": "[email protected]"
    }
  ]
}

Actual output

Without --flatten-components:

Matches expected output 👍

With --flatten-components:

{
  "components": [
    {
      "purl": "pkg:npm/[email protected]",
      "bom-ref": "[email protected]"
    },
    {
      "purl": "pkg:npm/[email protected]",
      "bom-ref": "[email protected]"
    },
    {
      "purl": "pkg:npm/[email protected]",
      "bom-ref": "[email protected]|[email protected]"
    }
    {
      "purl": "pkg:npm/[email protected]",
      "bom-ref": "[email protected]|[email protected]"
    }
  ]
}

[jkowalleck]

yes, this is intentional.
if one component appears multiple times in the SBOM, then because NPM actually installed it multiple times, or because it was a bundledDependency of any package.
To make this visible, this component appears multiple times in the SBOM result.
This is just how NPM works, and therefore it is rendered accordingly.

in your example the [email protected]|[email protected] is not the same as [email protected]|[email protected]. it might be equal, but this is still not guaranteed.

---

PS:
have a look at an example SBOM: https://github.com/CycloneDX/cyclonedx-node-npm/blob/main/demo/juice-shop/example-results/bom.1.4.xml
you will find [email protected] installed on top level, and multiple instances of [email protected] are installed in nested environments, and instances of [email protected] are installed in other nested environments.
NPM ACTUALLY installed multiple instances of the same version of the same dependency at several places in a way that makes them unique(not de-duplicated, not identical).
The actual dependency graph in the SBOM is a whole different topic.

see also the docs : https://github.com/CycloneDX/cyclonedx-node-npm/blob/main/docs/result.md

[rkg-mm]

Tools like Dependency-Track make this very visible to projects and I already know I will have to explain this to all projects, chief architect and some more people who will look at this from time to time...

[jkowalleck]

Please keep in mind: SBOM is about facts and evidences, not assumptions and hopes.

In the NPM context SBOM is not another representation of your package.json, it is the truth about your installed node-modules.
Just as i said... if a component appears multiple times, then for a good reason.

If NPM installs a package multiple times, it will appear in the SBOM multiple times.
And as long as you do not use the --flatten-components option, you will be able to tell where exactly it is in your file system by traversing the sub-components-tree of the SBOM.

[stevespringett]

I think the behavior of the prior node version was to NOT duplicate components. Most build plugins do not duplicate components, rather the component is listed only one time, but all the dependency relationships are captured correctly.

Duplicating components indicates quantity. For example, if a product ships with multiple occurances of jquery 2.0, then jquery 2.0 will be listed in the component inventory multiple times. If only a single occurance of jquery is distributed, then it would only appear once, regardless of how many other components depend on it. The quantity aspect will also be eliminated in CycloneDX v1.5, so in the future, there should be no reason to have duplicate components unless they have different data. For example, if both occurances of jquery had different hash values indicating one has been modified.

[jkowalleck]

@stevespringett i will wait for CDX 1.5 and re-visit this issue.
hopefully CycloneDX/specification#129 will bring the needed features

For now, I would argue, that if NPM installs multiple versions of jquery 2.0 in several places, then it will be shipped multiple times, right?
Therefore, the SBOM will have it listed multiple times -- as nested components just as NPM installed them.
Just like strip-ansi is listed multiple times (regardless of actual dependency-graph) in https://github.com/CycloneDX/cyclonedx-node-npm/blob/main/demo/juice-shop/example-results/bom.1.4.xml which is generated from https://github.com/CycloneDX/cyclonedx-node-npm/tree/main/demo/juice-shop/project.

$ cd demo/juice-shop/project
$ find -type d -name 'strip-ansi'
./node_modules/replace/node_modules/strip-ansi
./node_modules/cliui/node_modules/strip-ansi
./node_modules/yargs/node_modules/strip-ansi
./node_modules/wrap-ansi/node_modules/strip-ansi
./node_modules/strip-ansi
./node_modules/@mapbox/node-pre-gyp/node_modules/strip-ansi
./node_modules/node-gyp/node_modules/strip-ansi

[jkowalleck]

I think the behavior of the prior node version was to NOT duplicate components. Most build plugins do not duplicate components, rather the component is listed only one time, but all the dependency relationships are captured correctly.

tested it with the prior node version (@cyclonedx/bom@^3) on the same
setup from https://github.com/CycloneDX/cyclonedx-node-npm/tree/main/demo/juice-shop/project

result: no multiple occurrences of [email protected] appeared in the SBOM result.
which seams to be caused by a bug, not a feature. Furthermore, some used components were missing, and some dependencies to those missing instances of [email protected] were also missing.

Compare

• created with @cyclonedx-cyclonedx-npm - https://github.com/CycloneDX/cyclonedx-node-npm/blob/main/demo/juice-shop/example-results/bom.1.3.xml
• created with @cyclonedx/bom@^3 https://gist.github.com/jkowalleck/f37a9cb88c88c55e20be40a07e99e771

[jkowalleck]

@mmarseu @rkg-mm @stevespringett
please help to discuss how after-the-fact component de-duplication could be done;
threats and votes here: #307

i understand your need for such a feature, therefore https://github.com/CycloneDX/cyclonedx-node-npm/milestone/2 was created