-
-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for CEL policy conditions #316
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nscuro
force-pushed
the
cel-policies
branch
from
September 19, 2023 20:39
3b8cd5d
to
1bf6df0
Compare
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
…type Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
DataNucleus on its own loads too much data, and does so using too many queries. Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
…ustom functions Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Using `escapeJson` doesn't work quite right when special characters / regular expressions are provided. All we need is prevention of "breaking out" of strings, so escaping double quotes alone is sufficient. Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
* added tests for hash policy Signed-off-by: mehab <[email protected]> * updated tests Signed-off-by: mehab <[email protected]> --------- Signed-off-by: mehab <[email protected]>
* Add version cel policy script builder Signed-off-by: vithikashukla <[email protected]> * add version support for coordinates cel policy Signed-off-by: vithikashukla <[email protected]> * Added unit test for version policy script builder Signed-off-by: vithikashukla <[email protected]> * added coordninates condition test Signed-off-by: vithikashukla <[email protected]> * added coordinates condition test Signed-off-by: vithikashukla <[email protected]> --------- Signed-off-by: vithikashukla <[email protected]> Co-authored-by: vithikashukla <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
…EXT` Signed-off-by: nscuro <[email protected]>
Signed-off-by: nscuro <[email protected]>
Signed-off-by: mehab <[email protected]>
* Add version cel policy script builder Signed-off-by: vithikashukla <[email protected]> * add version support for coordinates cel policy Signed-off-by: vithikashukla <[email protected]> * Added unit test for version policy script builder Signed-off-by: vithikashukla <[email protected]> * added coordninates condition test Signed-off-by: vithikashukla <[email protected]> * added coordinates condition test Signed-off-by: vithikashukla <[email protected]> * added more conditions to test Signed-off-by: vithikashukla <[email protected]> * Added license condition test Signed-off-by: vithikashukla <[email protected]> * Update src/main/java/org/dependencytrack/policy/cel/CelPolicyEngine.java Co-authored-by: Niklas <[email protected]> Signed-off-by: VithikaS <[email protected]> * Added license group condition test Signed-off-by: vithikashukla <[email protected]> * updated comment Signed-off-by: vithikashukla <[email protected]> --------- Signed-off-by: vithikashukla <[email protected]> Signed-off-by: VithikaS <[email protected]> Co-authored-by: vithikashukla <[email protected]> Co-authored-by: Niklas <[email protected]>
nscuro
force-pushed
the
cel-policies
branch
from
September 26, 2023 09:24
d19301c
to
d6eb71b
Compare
Signed-off-by: nscuro <[email protected]>
Signed-off-by: vithikashukla <[email protected]>
…-policy support wildcard
Signed-off-by: nscuro <[email protected]>
As this feature will be backported, we need to make sure policies will be compatible once folks start upgrading to Hyades. Signed-off-by: nscuro <[email protected]>
nscuro
force-pushed
the
cel-policies
branch
from
September 26, 2023 18:56
d26815c
to
1d78e4b
Compare
Signed-off-by: nscuro <[email protected]>
Signed-off-by: Niklas <[email protected]>
VithikaS
approved these changes
Sep 27, 2023
mehab
pushed a commit
that referenced
this pull request
Sep 30, 2023
Signed-off-by: mehab <[email protected]> Add support for CEL policy conditions (#316) * Initial commit of CEL policy work Signed-off-by: nscuro <[email protected]> * Add a few custom CEL functions Signed-off-by: nscuro <[email protected]> * Make policies work with legacy way of reporting violations Signed-off-by: nscuro <[email protected]> * Implement `is_dependency_of` CEL function Signed-off-by: nscuro <[email protected]> * Support vuln aliases in CEL policies Signed-off-by: nscuro <[email protected]> * Few minor adjustments Signed-off-by: nscuro <[email protected]> * Return CEL errors in API response Signed-off-by: nscuro <[email protected]> * Fix some vulnerability fields not being fetched for policies Signed-off-by: nscuro <[email protected]> * Bump `versatile` to `0.3.0` Signed-off-by: nscuro <[email protected]> * Use AST visitor to determine which fields are accessed for any given type Signed-off-by: nscuro <[email protected]> * Cleanup Signed-off-by: nscuro <[email protected]> * Cleanup Signed-off-by: nscuro <[email protected]> * WIP: Loading of required fields; Project policy evaluation Signed-off-by: nscuro <[email protected]> * Improve violation reconciliation for projects Signed-off-by: nscuro <[email protected]> * Add test with bloated BOM to debug performance bottlenecks Signed-off-by: nscuro <[email protected]> * Disable DataNucleus L1 cache for policy reconciliation Signed-off-by: nscuro <[email protected]> * Add field mapping tests Signed-off-by: nscuro <[email protected]> * Handle implicit policy script requirements for custom functions Signed-off-by: nscuro <[email protected]> * Minor readability and code documentation improvements Signed-off-by: nscuro <[email protected]> * Fetch data for policy violation notifications in a single query DataNucleus on its own loads too much data, and does so using too many queries. Signed-off-by: nscuro <[email protected]> * Perform violation reconciliation using direct JDBC access Signed-off-by: nscuro <[email protected]> * Include strings library in CEL policy environment Signed-off-by: nscuro <[email protected]> * Cleanup; Support project properties, tags, and vulnerability aliases Signed-off-by: nscuro <[email protected]> * Add test to verify that all fields can be loaded Signed-off-by: nscuro <[email protected]> * Add remaining fields to `testWithAllFields` Signed-off-by: nscuro <[email protected]> * Add test for vuln severity evaluation Signed-off-by: nscuro <[email protected]> * Remove un-implemented `depends_on` function; Add proper logging for custom functions Signed-off-by: nscuro <[email protected]> * Handle invalid scripts and script runtime failures Signed-off-by: nscuro <[email protected]> * Add `escapeQuotes` for CEL script builders Using `escapeJson` doesn't work quite right when special characters / regular expressions are provided. All we need is prevention of "breaking out" of strings, so escaping double quotes alone is sufficient. Signed-off-by: nscuro <[email protected]> * Add tests for some legacy conditions Signed-off-by: nscuro <[email protected]> * More tests for `CelPolicyEngine` Signed-off-by: nscuro <[email protected]> * Add more tests; Implement script cache bypass for REST API interactions Signed-off-by: nscuro <[email protected]> * Add tests for hash policy (#326) * added tests for hash policy Signed-off-by: mehab <[email protected]> * updated tests Signed-off-by: mehab <[email protected]> --------- Signed-off-by: mehab <[email protected]> * Add version cel policy script builder (#324) * Add version cel policy script builder Signed-off-by: vithikashukla <[email protected]> * add version support for coordinates cel policy Signed-off-by: vithikashukla <[email protected]> * Added unit test for version policy script builder Signed-off-by: vithikashukla <[email protected]> * added coordninates condition test Signed-off-by: vithikashukla <[email protected]> * added coordinates condition test Signed-off-by: vithikashukla <[email protected]> --------- Signed-off-by: vithikashukla <[email protected]> Co-authored-by: vithikashukla <[email protected]> * Fix new UNIQUE constraint breaking existing behavior Signed-off-by: nscuro <[email protected]> * Add feature flag for CEL policy engine Signed-off-by: nscuro <[email protected]> * Add `UpgradeItem` to update type of `"POLICYCONDITION"."VALUE"` to `TEXT` Signed-off-by: nscuro <[email protected]> * Handle policy evaluation for individual components Signed-off-by: nscuro <[email protected]> * added unit tests for cwe cel policy Signed-off-by: mehab <[email protected]> * Add license condition test (#332) * Add version cel policy script builder Signed-off-by: vithikashukla <[email protected]> * add version support for coordinates cel policy Signed-off-by: vithikashukla <[email protected]> * Added unit test for version policy script builder Signed-off-by: vithikashukla <[email protected]> * added coordninates condition test Signed-off-by: vithikashukla <[email protected]> * added coordinates condition test Signed-off-by: vithikashukla <[email protected]> * added more conditions to test Signed-off-by: vithikashukla <[email protected]> * Added license condition test Signed-off-by: vithikashukla <[email protected]> * Update src/main/java/org/dependencytrack/policy/cel/CelPolicyEngine.java Co-authored-by: Niklas <[email protected]> Signed-off-by: VithikaS <[email protected]> * Added license group condition test Signed-off-by: vithikashukla <[email protected]> * updated comment Signed-off-by: vithikashukla <[email protected]> --------- Signed-off-by: vithikashukla <[email protected]> Signed-off-by: VithikaS <[email protected]> Co-authored-by: vithikashukla <[email protected]> Co-authored-by: Niklas <[email protected]> * Fix projection mapping for `Double` / `BigDecimal` fields Signed-off-by: nscuro <[email protected]> * support wildcard Signed-off-by: vithikashukla <[email protected]> * Add `buf` config and workflow Signed-off-by: nscuro <[email protected]> * Change Proto package from `hyades` to `dependencytrack` As this feature will be backported, we need to make sure policies will be compatible once folks start upgrading to Hyades. Signed-off-by: nscuro <[email protected]> * Fix failing tests due to Proto package change Signed-off-by: nscuro <[email protected]> * Un-ignore `cyclonedx.proto` from breaking changes check Signed-off-by: Niklas <[email protected]> --------- Signed-off-by: nscuro <[email protected]> Signed-off-by: mehab <[email protected]> Signed-off-by: vithikashukla <[email protected]> Signed-off-by: VithikaS <[email protected]> Signed-off-by: Niklas <[email protected]> Co-authored-by: meha <[email protected]> Co-authored-by: VithikaS <[email protected]> Co-authored-by: vithikashukla <[email protected]> Co-authored-by: mehab <[email protected]> changes for sending repo meta analysis events from apiserver Signed-off-by: mehab <[email protected]> initial refactoring Signed-off-by: mehab <[email protected]> refactored code Signed-off-by: mehab <[email protected]> code changes completed for sending Signed-off-by: mehab <[email protected]> fixed component resource unit tests Signed-off-by: mehab <[email protected]>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR adds support for policy conditions using the Common Expression Language (CEL), as originally brought up in DependencyTrack/dependency-track#2673.
Corresponding frontend changes are here: https://github.com/nscuro/dependency-track-frontend/tree/cel-policies-hyades
TODOs
CelPolicyScriptSourceBuilder
s for compatibility with legacy conditions:ComponentAgeImplement tests for allCelPolicyScriptSourceBuilder
sCelPolicyEngine
that was introduced during initial implementationCelPolicyEngine
, that exercises:CelPolicyLibrary
policy.proto
@nscuro Write documentation, including:Overview of CELWhat variables, fields, functions, etc. are availableExample policieshyades
repositoryAddressed Issue
N/A
Additional Details
CEL Language Definition: https://github.com/google/cel-spec/blob/master/doc/langdef.md
CEL-Java implementation by Project Nessie: https://github.com/projectnessie/cel-java
Example expressions
Behavior changes over legacy policy engine
SeverityPolicyEvaluator
andVulnerabilityIdPolicyEvaluator
could yield multiple violations; One for each matched vulnerability. Because violations are attached to components, not individual vulnerabilities, this behavior did not make sense. It is thus not carried over to the CEL policy engine.Checklist
This PR fixes a defect, and I have provided tests to verify that the fix is effectiveThis PR introduces changes to the database model, and I have added corresponding update logicThis PR introduces new or alters existing behavior, and I have updated the documentation accordingly