Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/integrity analysis apiserver #336

Closed
wants to merge 7 commits into from

Conversation

mehab
Copy link
Collaborator

@mehab mehab commented Sep 28, 2023

Description

Addressed Issue

Additional Details

Checklist

  • I have read and understand the contributing guidelines
  • This PR fixes a defect, and I have provided tests to verify that the fix is effective
  • This PR implements an enhancement, and I have provided tests to verify that it works as intended
  • This PR introduces changes to the database model, and I have added corresponding update logic
  • This PR introduces new or alters existing behavior, and I have updated the documentation accordingly

nscuro and others added 5 commits September 30, 2023 07:48
Signed-off-by: mehab <[email protected]>
Add support for CEL policy conditions (#316)

* Initial commit of CEL policy work

Signed-off-by: nscuro <[email protected]>

* Add a few custom CEL functions

Signed-off-by: nscuro <[email protected]>

* Make policies work with legacy way of reporting violations

Signed-off-by: nscuro <[email protected]>

* Implement `is_dependency_of` CEL function

Signed-off-by: nscuro <[email protected]>

* Support vuln aliases in CEL policies

Signed-off-by: nscuro <[email protected]>

* Few minor adjustments

Signed-off-by: nscuro <[email protected]>

* Return CEL errors in API response

Signed-off-by: nscuro <[email protected]>

* Fix some vulnerability fields not being fetched for policies

Signed-off-by: nscuro <[email protected]>

* Bump `versatile` to `0.3.0`

Signed-off-by: nscuro <[email protected]>

* Use AST visitor to determine which fields are accessed for any given type

Signed-off-by: nscuro <[email protected]>

* Cleanup

Signed-off-by: nscuro <[email protected]>

* Cleanup

Signed-off-by: nscuro <[email protected]>

* WIP: Loading of required fields; Project policy evaluation

Signed-off-by: nscuro <[email protected]>

* Improve violation reconciliation for projects

Signed-off-by: nscuro <[email protected]>

* Add test with bloated BOM to debug performance bottlenecks

Signed-off-by: nscuro <[email protected]>

* Disable DataNucleus L1 cache for policy reconciliation

Signed-off-by: nscuro <[email protected]>

* Add field mapping tests

Signed-off-by: nscuro <[email protected]>

* Handle implicit policy script requirements for custom functions

Signed-off-by: nscuro <[email protected]>

* Minor readability and code documentation improvements

Signed-off-by: nscuro <[email protected]>

* Fetch data for policy violation notifications in a single query

DataNucleus on its own loads too much data, and does so using too many queries.

Signed-off-by: nscuro <[email protected]>

* Perform violation reconciliation using direct JDBC access

Signed-off-by: nscuro <[email protected]>

* Include strings library in CEL policy environment

Signed-off-by: nscuro <[email protected]>

* Cleanup; Support project properties, tags, and vulnerability aliases

Signed-off-by: nscuro <[email protected]>

* Add test to verify that all fields can be loaded

Signed-off-by: nscuro <[email protected]>

* Add remaining fields to `testWithAllFields`

Signed-off-by: nscuro <[email protected]>

* Add test for vuln severity evaluation

Signed-off-by: nscuro <[email protected]>

* Remove un-implemented `depends_on` function; Add proper logging for custom functions

Signed-off-by: nscuro <[email protected]>

* Handle invalid scripts and script runtime failures

Signed-off-by: nscuro <[email protected]>

* Add `escapeQuotes` for CEL script builders

Using `escapeJson` doesn't work quite right when special characters / regular expressions are provided. All we need is prevention of "breaking out" of strings, so escaping double quotes alone is sufficient.

Signed-off-by: nscuro <[email protected]>

* Add tests for some legacy conditions

Signed-off-by: nscuro <[email protected]>

* More tests for `CelPolicyEngine`

Signed-off-by: nscuro <[email protected]>

* Add more tests; Implement script cache bypass for REST API interactions

Signed-off-by: nscuro <[email protected]>

* Add tests for hash policy (#326)

* added tests for hash policy

Signed-off-by: mehab <[email protected]>

* updated tests

Signed-off-by: mehab <[email protected]>

---------

Signed-off-by: mehab <[email protected]>

* Add version cel policy script builder (#324)

* Add version cel policy script builder

Signed-off-by: vithikashukla <[email protected]>

* add version support for coordinates cel policy

Signed-off-by: vithikashukla <[email protected]>

* Added unit test for version policy script builder

Signed-off-by: vithikashukla <[email protected]>

* added coordninates condition test

Signed-off-by: vithikashukla <[email protected]>

* added coordinates condition test

Signed-off-by: vithikashukla <[email protected]>

---------

Signed-off-by: vithikashukla <[email protected]>
Co-authored-by: vithikashukla <[email protected]>

* Fix new UNIQUE constraint breaking existing behavior

Signed-off-by: nscuro <[email protected]>

* Add feature flag for CEL policy engine

Signed-off-by: nscuro <[email protected]>

* Add `UpgradeItem` to update type of `"POLICYCONDITION"."VALUE"` to `TEXT`

Signed-off-by: nscuro <[email protected]>

* Handle policy evaluation for individual components

Signed-off-by: nscuro <[email protected]>

* added unit tests for cwe cel policy

Signed-off-by: mehab <[email protected]>

* Add license condition test (#332)

* Add version cel policy script builder

Signed-off-by: vithikashukla <[email protected]>

* add version support for coordinates cel policy

Signed-off-by: vithikashukla <[email protected]>

* Added unit test for version policy script builder

Signed-off-by: vithikashukla <[email protected]>

* added coordninates condition test

Signed-off-by: vithikashukla <[email protected]>

* added coordinates condition test

Signed-off-by: vithikashukla <[email protected]>

* added more conditions to test

Signed-off-by: vithikashukla <[email protected]>

* Added license condition test

Signed-off-by: vithikashukla <[email protected]>

* Update src/main/java/org/dependencytrack/policy/cel/CelPolicyEngine.java

Co-authored-by: Niklas <[email protected]>
Signed-off-by: VithikaS <[email protected]>

* Added license group condition test

Signed-off-by: vithikashukla <[email protected]>

* updated comment

Signed-off-by: vithikashukla <[email protected]>

---------

Signed-off-by: vithikashukla <[email protected]>
Signed-off-by: VithikaS <[email protected]>
Co-authored-by: vithikashukla <[email protected]>
Co-authored-by: Niklas <[email protected]>

* Fix projection mapping for `Double` / `BigDecimal` fields

Signed-off-by: nscuro <[email protected]>

* support wildcard

Signed-off-by: vithikashukla <[email protected]>

* Add `buf` config and workflow

Signed-off-by: nscuro <[email protected]>

* Change Proto package from `hyades` to `dependencytrack`

As this feature will be backported, we need to make sure policies will be compatible once folks start upgrading to Hyades.

Signed-off-by: nscuro <[email protected]>

* Fix failing tests due to Proto package change

Signed-off-by: nscuro <[email protected]>

* Un-ignore `cyclonedx.proto` from breaking changes check

Signed-off-by: Niklas <[email protected]>

---------

Signed-off-by: nscuro <[email protected]>
Signed-off-by: mehab <[email protected]>
Signed-off-by: vithikashukla <[email protected]>
Signed-off-by: VithikaS <[email protected]>
Signed-off-by: Niklas <[email protected]>
Co-authored-by: meha <[email protected]>
Co-authored-by: VithikaS <[email protected]>
Co-authored-by: vithikashukla <[email protected]>
Co-authored-by: mehab <[email protected]>

changes for sending repo meta analysis events from apiserver

Signed-off-by: mehab <[email protected]>

initial refactoring

Signed-off-by: mehab <[email protected]>

refactored code

Signed-off-by: mehab <[email protected]>

code changes completed for sending

Signed-off-by: mehab <[email protected]>

fixed component resource unit tests

Signed-off-by: mehab <[email protected]>
Signed-off-by: vithikashukla <[email protected]>
@mehab mehab force-pushed the feature/integrityAnalysisApiserver branch from a207295 to 6fb17ce Compare September 30, 2023 06:54
@mehab mehab closed this Oct 2, 2023
@mehab
Copy link
Collaborator Author

mehab commented Oct 2, 2023

closed as branch history became muddled

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 1, 2023
@mehab mehab deleted the feature/integrityAnalysisApiserver branch February 21, 2024 09:44
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants