Ecosystem license strings are not always valid SPDX license expressions

[goneall]

When running Trivy ecosystems enrich [file] the resultant SPDX document will, on occasion, result in a concluded license (in the SPDX format) that does not validate.

From looking at the code, it looks like the license data originates from the ecosystems.me packages API.

My guess is that ecosystem.me is just pulling the raw data from the packages metadata files and passing it along.

Some of the package managers, such as NPM, do a great job of complying with SPDX license IDs and expressions. Others, such as Maven, accept just about any string.

I'm not sure if this is an issue we should tackle in Parlay or in the upstream ecosystems.me packages. Since Parlay is producing SBOMs which specify SPDX license expressions (for both SPDX and CDX) and the upstream doesn't claim to comply with the standard, it may be best to fix it in this library.

Here's some code I wrote that fixes this issue downstream in a Python application for the SPDX standard.

The basic approach is to:

• Detect if the license is valid using a license expression parser
• If it is not valid, create a LicenseRef- or ExtractedLicenseInfo in the SBOM to capture the original string
• Replace the concluded license with the LicenseRef-

Since I have a downstream solution, no urgency on a solution - but if this is something the maintainers would like to fix, I'd be happy to help out. I'm not much of a Golang programmer (I've written less than 10 lines of code so far), but I can help with the algorithms and SPDX spec.

[mcombuechen]

Hey @goneall
thanks for pointing this out. It sounds like a straightforward fix, but really the data source (ecosyste.ms) should sanitize license identifiers. But I can see how that's not necessarily feasible, there might always be licenses that don't fit in the official SPDX set.
As I'm a bit swamped at the moment, I won't have time to implement this myself. More likely in a few weeks.

[goneall]

Thanks @mcombuechen for the reply - no hurry from my perspective as I have a fix downstream from Parlay.

I'd be happy to help from a review / spec perspective - since I'm very new to Golang (having written a total of 5 lines of code so far), it would probably be more work for you to review / correct my attempt at a PR than to implement it yourself ;).