-
Notifications
You must be signed in to change notification settings - Fork 359
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: support evaluating spdx license expressions #1329
base: main
Are you sure you want to change the base?
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1329 +/- ##
==========================================
+ Coverage 68.43% 68.69% +0.26%
==========================================
Files 183 184 +1
Lines 17606 17756 +150
==========================================
+ Hits 12049 12198 +149
- Misses 4895 4896 +1
Partials 662 662 ☔ View full report in Codecov by Sentry. |
45210db
to
30babc5
Compare
30babc5
to
9bebbe3
Compare
Hi , I'd love to see this progress. We are already using osv-scanner in one small private repo replacing a fork of https://github.com/jslicense/licensee.js that adds pnpm support, and would like to continue the transition but this PR is needed to avoid adding a ridiculous number of ignores in our |
This implements a parser for SPDX license expressions in accordance with annex D of the v2 spec, to allow the scanner to properly determine if packages with such expressive licenses are permitted based on the licenses allowed by the user.
To do this I've implemented a two-phase parser which starts by tokenizing the license string and then turns it into an AST of nodes that can be walked to determine if the full expression is satisfied; for no particularly good reason I have used a string for the base token type rather than a
struct
, meaning the tokens value is also it's type - the tradeoff with this is while it means we don't have to do as much referencing or work in the tokenizer, we do have to do some extra work when walking the tree to resolve the "simple expression" tokens.I'm proposing landing the current implementation as I don't think using a
struct
would be strictly better, though in hindsight it probably would have been a bit quicker to implement and so I plan to explore how much simpler (or complex) it might be as a follow up.Currently this is focused on
AND
andOR
support, as I believe those are the two primary operators that are relevant to the scanner, though we still might want to have richer handling for theWITH
and+
operators; currently both of those just get treated as being part of the license expression (though it's not actually possible right now to allow a license withWITH
as the CLI expects license values to not have any spaces - this too will be a follow-up for me).Finally, I've purposely not put any caching in place even though that should be easy, due to wanting to get this landed and as I don't expect it to actually have a significant impact on the scanner performance (ultimately most complex expressions in the real-world will be made up of a single operator, and chopping+looping over strings in memory is extremely fast)
Resolves #1299