New plugin API #9002

mnonnenmacher · 2024-08-19T10:26:00Z

A new plugin API that has a model for plugin options and uses Kotlin symbol processing to reduce the amount of handwritten boilerplate code. See the commit messages and especially the new plugins/api/README.md file for details.

advisor/src/main/kotlin/AdviceProvider.kt

advisor/src/main/kotlin/AdviceProviderFactory.kt

codecov · 2024-08-19T10:44:43Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 68.02%. Comparing base (16a121c) to head (7455d2e).
Report is 12 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #9002   +/-   ##
=========================================
  Coverage     68.02%   68.02%           
  Complexity     1158     1158           
=========================================
  Files           241      241           
  Lines          7691     7691           
  Branches        867      867           
=========================================
  Hits           5232     5232           
  Misses         2103     2103           
  Partials        356      356

Flag	Coverage Δ
funTest-docker	`67.39% <ø> (ø)`
funTest-non-docker	`33.92% <ø> (ø)`
test	`37.92% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

plugins/api/src/main/kotlin/PluginFactoryGenerator.kt

sschuberth · 2024-08-25T13:28:15Z

BTW, one thing that came to my mind: If writing plugins now requires a Gradle plugin, we'd also need to publish that plugin in order to allow third party authors to create ORT plugins.

mnonnenmacher · 2024-08-25T17:46:57Z

BTW, one thing that came to my mind: If writing plugins now requires a Gradle plugin, we'd also need to publish that plugin in order to allow third party authors to create ORT plugins.

It only requires the KSP plugin, no custom plugin. Also, the boilerplate code could still be written by hand.

plugins/api/src/main/kotlin/PluginProcessor.kt

plugins/advisors/nexus-iq/src/main/kotlin/NexusIq.kt

sschuberth · 2024-08-27T17:42:07Z

clients/osv/src/main/kotlin/OsvService.kt

-         * Create a service instance for communicating with the given [server], optionally using a pre-built OkHttp
-         * [client].
-         */
-        fun create(server: Server, client: OkHttpClient? = null): OsvService = create(server.url, client)


Hmm, does this removal make the following implementations easier? Because if not, I'd prefer to keep it, and later on refactor it differently, to have functions with the following signatures (and do the same for ClearlyDefinedService):

fun create(server: Server = Server.PRODUCTION, client: OkHttpClient? = null): OsvService fun create(url: String, client: OkHttpClient? = null): OsvService

Because to me it makes more sense to use the URL constructor only in cases where you want a non-default URL.

Well, it's not used anywhere and I don't think the OSV advisor which is the only consumer of the library would ever use it. However, the important part is happening in the next commit and moving the URL value to a constant could happen without removing the enum. It just appeared to me that removing unused code along the way would be a good idea.

clients/osv/src/main/kotlin/OsvService.kt

plugins/api/README.md

advisor/src/main/kotlin/AdviceProvider.kt

plugins/advisors/build.gradle.kts

plugins/api/README.md

sschuberth · 2024-08-28T12:30:54Z

I believe in OsvFunTest now some "OSV" vs "Osv" adjustments are necessary in the name of the AdvisorDetails.

Signed-off-by: Martin Nonnenmacher <[email protected]>

Convert `Server.PRODUCTION` to a compile constant and remove the unused `STAGING` constant to simplify code. Signed-off-by: Martin Nonnenmacher <[email protected]>

Add a new plugin API with the following features: * Plugins have a `PluginDescriptor` which holds the metadata of the plugin. Besides the name this also holds a list of the supported plugin configuration options and their types. This can be used by the CLI and other tools like the ORT server to show the supported options to the user. * Boilerplate code like the plugin factory implementation which includes creating the configuration object from the plain options and secrets maps can be generated by a Kotlin symbol processor. To mark plugin classes for the symbol processor, the new `@OrtPlugin` annotation is introduced. * Plugin configuration classes must use only properties with `String`, `Int`, `Long`, `Boolean`, `Secret`, or `List<String>` type. To simplify the migration, the new Plugin API is created completely separate from the old API in a new `:plugins:api` module. Signed-off-by: Martin Nonnenmacher <[email protected]>

Migrate the Advisor API and all plugins to the new plugin API. This allows to simplify the code in several places, as the plugin factories and service loader classes do not have to be created by hand anymore. Signed-off-by: Martin Nonnenmacher <[email protected]>

Introduce an intermediary `PluginSpec` model that is used as input for the code generation and move the code to create a `PluginSpec` to the new class `PluginSpecFactory`. This provides a better separation of concerns and makes it easier to generate other output files based on the `PluginSpec`. Signed-off-by: Martin Nonnenmacher <[email protected]>

Generate a JSON representation of the plugin spec which can be used as input for other tools, for example, to generate documentation. Signed-off-by: Martin Nonnenmacher <[email protected]>

Move the KSP compiler to a separate project to better separate the plugin API from the compiler plugin. Signed-off-by: Martin Nonnenmacher <[email protected]>

Rename `ort-plugins-conventions` to `ort-plugin-parent-conventions` to make it clearer that it is used for parent projects of plugin projects. This avoids confusion when a new convention for plugin projects is introduced in the next commit. Signed-off-by: Martin Nonnenmacher <[email protected]>

Introduce the new `ort-plugin-conventions` plugin to simplify the configuration of plugin projects. Signed-off-by: Martin Nonnenmacher <[email protected]>

Signed-off-by: Martin Nonnenmacher <[email protected]>

Make clear that the config argument is optional. Signed-off-by: Martin Nonnenmacher <[email protected]>

mnonnenmacher · 2024-08-28T14:12:23Z

If this gets approved and tests pass I would merge it right after tomorrow's release. Then we can do more testing and adjustments, and maybe also migrate other plugins before the release next week.

github-advanced-security bot found potential problems Aug 19, 2024

View reviewed changes

mnonnenmacher force-pushed the new-plugin-api branch from 7ccaa75 to aced2fd Compare August 25, 2024 11:52

github-advanced-security bot found potential problems Aug 25, 2024

View reviewed changes

plugins/api/src/main/kotlin/PluginFactoryGenerator.kt Fixed Show fixed Hide fixed

plugins/api/src/main/kotlin/PluginFactoryGenerator.kt Fixed Show fixed Hide fixed

mnonnenmacher force-pushed the new-plugin-api branch from aced2fd to 33d52b2 Compare August 25, 2024 17:45

github-advanced-security bot found potential problems Aug 25, 2024

View reviewed changes

plugins/api/src/main/kotlin/PluginProcessor.kt Fixed Show fixed Hide fixed

plugins/api/src/main/kotlin/PluginProcessor.kt Fixed Show fixed Hide fixed

mnonnenmacher force-pushed the new-plugin-api branch from 33d52b2 to ce7206a Compare August 26, 2024 17:46

github-advanced-security bot found potential problems Aug 26, 2024

View reviewed changes

plugins/api/src/main/kotlin/PluginProcessor.kt Fixed Show fixed Hide fixed

mnonnenmacher force-pushed the new-plugin-api branch from ce7206a to 6be1869 Compare August 27, 2024 16:02

github-advanced-security bot found potential problems Aug 27, 2024

View reviewed changes

plugins/advisors/nexus-iq/src/main/kotlin/NexusIq.kt Fixed Show fixed Hide fixed

mnonnenmacher force-pushed the new-plugin-api branch from 6be1869 to 14b7b43 Compare August 27, 2024 17:28

mnonnenmacher marked this pull request as ready for review August 27, 2024 17:28

mnonnenmacher requested review from MarcelBochtler, oheger-bosch and a team as code owners August 27, 2024 17:28

mnonnenmacher force-pushed the new-plugin-api branch 2 times, most recently from 4382861 to bbb8dec Compare August 27, 2024 19:30

sschuberth requested changes Aug 27, 2024

View reviewed changes

mnonnenmacher force-pushed the new-plugin-api branch 2 times, most recently from 1deef45 to 1009025 Compare August 28, 2024 09:33

mnonnenmacher requested review from sschuberth and fviernau August 28, 2024 09:34

mnonnenmacher force-pushed the new-plugin-api branch from 1009025 to 1d16944 Compare August 28, 2024 11:25

mnonnenmacher added 3 commits August 28, 2024 16:10

refactor(osv-client)!: Remove an unused constructor

54f30ce

Signed-off-by: Martin Nonnenmacher <[email protected]>

refactor(osv-client)!: Remove the Server enum

29ac2e0

Convert `Server.PRODUCTION` to a compile constant and remove the unused `STAGING` constant to simplify code. Signed-off-by: Martin Nonnenmacher <[email protected]>

mnonnenmacher added 8 commits August 28, 2024 16:10

feat(plugins-api): Generate a JSON representation of the plugin spec

cb31f9c

Generate a JSON representation of the plugin spec which can be used as input for other tools, for example, to generate documentation. Signed-off-by: Martin Nonnenmacher <[email protected]>

refactor(plugins): Move KSP compiler to separate project

fb9a6f5

Move the KSP compiler to a separate project to better separate the plugin API from the compiler plugin. Signed-off-by: Martin Nonnenmacher <[email protected]>

build: Introduce a convention plugin for plugins

8628934

Introduce the new `ort-plugin-conventions` plugin to simplify the configuration of plugin projects. Signed-off-by: Martin Nonnenmacher <[email protected]>

refactor(compiler): Use singleOrNull() to simplify code

71bfe29

Signed-off-by: Martin Nonnenmacher <[email protected]>

fix(compiler): Fix an error message

7455d2e

Make clear that the config argument is optional. Signed-off-by: Martin Nonnenmacher <[email protected]>

mnonnenmacher force-pushed the new-plugin-api branch from 1d16944 to 7455d2e Compare August 28, 2024 14:10

sschuberth approved these changes Aug 28, 2024

View reviewed changes

mnonnenmacher merged commit ed095a6 into main Aug 29, 2024
23 checks passed

mnonnenmacher deleted the new-plugin-api branch August 29, 2024 08:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New plugin API #9002

New plugin API #9002

mnonnenmacher commented Aug 19, 2024 •

edited

Loading

codecov bot commented Aug 19, 2024 •

edited

Loading

sschuberth commented Aug 25, 2024

mnonnenmacher commented Aug 25, 2024 •

edited

Loading

sschuberth Aug 27, 2024

mnonnenmacher Aug 28, 2024

sschuberth commented Aug 28, 2024

mnonnenmacher commented Aug 28, 2024

New plugin API #9002

New plugin API #9002

Conversation

mnonnenmacher commented Aug 19, 2024 • edited Loading

codecov bot commented Aug 19, 2024 • edited Loading

Codecov Report

sschuberth commented Aug 25, 2024

mnonnenmacher commented Aug 25, 2024 • edited Loading

sschuberth Aug 27, 2024

Choose a reason for hiding this comment

mnonnenmacher Aug 28, 2024

Choose a reason for hiding this comment

sschuberth commented Aug 28, 2024

mnonnenmacher commented Aug 28, 2024

mnonnenmacher commented Aug 19, 2024 •

edited

Loading

codecov bot commented Aug 19, 2024 •

edited

Loading

mnonnenmacher commented Aug 25, 2024 •

edited

Loading