Support installing plugin SNAPSHOTs with SNASPHOT distribution #16581

reta · 2024-11-06T14:15:35Z

Description

Support installing plugin SNAPSHOTs with SNASPHOT distribution

/opensearch-2.19.0-SNAPSHOT$ ./bin/opensearch-plugin install transport-reactor-netty4
-> Installing transport-reactor-netty4                                                                                                                                     
-> Downloading transport-reactor-netty4 from opensearch                                                                                                                    
[=================================================] 100%

Related Issues

Part of opensearch-project/opensearch-build#5096, addresses #16219

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

reta · 2024-11-06T14:17:07Z

distribution/tools/plugin-cli/src/main/java/org/opensearch/plugins/InstallPluginCommand.java

                pluginId,
-                version,
-                stagingHash
+                version + "-SNAPSHOT"


This is tricky one: OpenSearch version has no SNAPSHOT suffix, but Build.CURRENT does

Couldn't we use the same as release distributions Build.CURRENT.getQualifiedVersion() ?
Sounds like the only diff is the artifacts URL, could we consolidate the if-else ?

Thanks @saratvemulapalli

This is tricky one: OpenSearch version has no SNAPSHOT suffix, but Build.CURRENT does

We don't pass build version here, only the version (which is current) + isSnapshot, I think it is fine (vs adding additional arguments)

Sounds like the only diff is the artifacts URL, could we consolidate the if-else ?

This if-else is only about artifacts URL (baseUrl)

reta · 2024-11-06T14:20:17Z

The signature part is not yet settled:

/opensearch-2.19.0-SNAPSHOT$ ./bin/opensearch-plugin install transport-reactor-netty4                                                                                                                                                                                                     
-> Installing transport-reactor-netty4                                                                                                                                     
-> Downloading transport-reactor-netty4 from opensearch                                                                                                                    
[=================================================] 100%                                                                                                                   
-> Failed installing transport-reactor-netty4                                                                                                                              
-> Rolling back transport-reactor-netty4                                                                                                                                   
-> Rolled back transport-reactor-netty4                                                                                                                                    
Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL: https://artifacts.opensearch.org/snapshots/plugins/transport-reactor-netty4/2.19.0-SNAPSHOT/transport-reactor-netty4-2.19.0-SNAPSHOT.zip.sig
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:2013)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1614)
        at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:223)
        at java.base/java.net.URL.openStream(URL.java:1325)                                                                                                                
        at org.opensearch.plugins.InstallPluginCommand.urlOpenStream(InstallPluginCommand.java:490)
        at org.opensearch.plugins.InstallPluginCommand.verifySignature(InstallPluginCommand.java:630)
        at org.opensearch.plugins.InstallPluginCommand.downloadAndValidate(InstallPluginCommand.java:608)
        at org.opensearch.plugins.InstallPluginCommand.download(InstallPluginCommand.java:316)
        at org.opensearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:271)
        at org.opensearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:248)
        at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
        at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
        at org.opensearch.cli.MultiCommand.execute(MultiCommand.java:104)
        at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
        at org.opensearch.cli.Command.main(Command.java:101)
        at org.opensearch.plugins.PluginCli.main(PluginCli.java:60)

github-actions · 2024-11-06T14:38:00Z

❌ Gradle check result for 6268761: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2024-11-06T15:09:21Z

❌ Gradle check result for 422c83e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

distribution/tools/plugin-cli/src/main/java/org/opensearch/plugins/InstallPluginCommand.java

github-actions · 2024-11-06T15:53:48Z

❌ Gradle check result for 65d5cc9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2024-11-06T16:39:52Z

❌ Gradle check result for e7a48c0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

distribution/tools/plugin-cli/src/main/java/org/opensearch/plugins/InstallPluginCommand.java

github-actions · 2024-11-06T17:41:16Z

✅ Gradle check result for 1774f1d: SUCCESS

codecov · 2024-11-06T17:42:41Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.07%. Comparing base (e688388) to head (7b24470).
Report is 5 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #16581      +/-   ##
============================================
- Coverage     72.15%   72.07%   -0.08%     
+ Complexity    65128    65055      -73     
============================================
  Files          5315     5315              
  Lines        303573   303568       -5     
  Branches      43925    43924       -1     
============================================
- Hits         219036   218799     -237     
- Misses        66565    66795     +230     
- Partials      17972    17974       +2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-actions · 2024-11-08T14:12:42Z

❌ Gradle check result for 1255cf8: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Andriy Redko <[email protected]>

github-actions · 2024-11-08T16:51:52Z

❕ Gradle check result for 7b24470: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

reta · 2024-11-11T21:32:30Z

Success!

$ ./bin/opensearch-plugin install transport-nio
-> Installing transport-nio
-> Downloading transport-nio from opensearch
[=================================================] 100%   
-> Installed transport-nio with folder name transport-nio

reta · 2024-11-12T19:17:56Z

$ /home/opensearch-3.0.0-SNAPSHOT$ ./bin/opensearch-plugin install transport-nio
-> Installing transport-nio
-> Downloading transport-nio from opensearch
[=================================================] 100%   
-> Installed transport-nio with folder name transport-nio

$ ll plugins/transport-nio/
total 3.3M
-rw-r--r-- 1 user group  12K Nov 12 14:15 LICENSE.txt
-rw-r--r-- 1 user group 329K Nov 12 14:15 netty-buffer-4.1.114.Final.jar
-rw-r--r-- 1 user group 344K Nov 12 14:15 netty-codec-4.1.114.Final.jar
-rw-r--r-- 1 user group 653K Nov 12 14:15 netty-codec-http-4.1.114.Final.jar
-rw-r--r-- 1 user group 679K Nov 12 14:15 netty-common-4.1.114.Final.jar
-rw-r--r-- 1 user group 564K Nov 12 14:15 netty-handler-4.1.114.Final.jar
-rw-r--r-- 1 user group  37K Nov 12 14:15 netty-resolver-4.1.114.Final.jar
-rw-r--r-- 1 user group 510K Nov 12 14:15 netty-transport-4.1.114.Final.jar
-rw-r--r-- 1 user group  44K Nov 12 14:15 netty-transport-native-unix-common-4.1.114.Final.jar
-rw-r--r-- 1 user group  16K Nov 12 14:15 NOTICE.txt
-rw-r--r-- 1 user group  59K Nov 12 14:15 opensearch-nio-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 user group 1.9K Nov 12 14:15 plugin-descriptor.properties
-rw-r--r-- 1 user group 1.7K Nov 12 14:15 plugin-security.policy
-rw-r--r-- 1 user group  66K Nov 12 14:15 transport-nio-client-3.0.0-SNAPSHOT.jar

reta · 2024-11-13T13:36:35Z

@saratvemulapalli LGTY? thanks!

reta added the skip-changelog label Nov 6, 2024

reta requested review from anasalkouz, andrross, ashking94, Bukhtawar, CEHENKLE, dblock, dbwiddis, gbbafna, jainankitk, kotwanikunal, linuxpi, mch2, msfroh, nknize, owaiskazi19, Rishikesh1159, sachinpkale, saratvemulapalli, shwetathareja, sohami and VachaShah as code owners November 6, 2024 14:15

reta commented Nov 6, 2024

View reviewed changes

reta force-pushed the issue-build-5096 branch from 13e4822 to 6268761 Compare November 6, 2024 14:23

reta mentioned this pull request Nov 6, 2024

Add pluginZip publication for plugins in the plugins folder #16219

Closed

3 tasks

reta force-pushed the issue-build-5096 branch from 6268761 to 422c83e Compare November 6, 2024 14:48

rursprung reviewed Nov 6, 2024

View reviewed changes

distribution/tools/plugin-cli/src/main/java/org/opensearch/plugins/InstallPluginCommand.java Outdated Show resolved Hide resolved

reta force-pushed the issue-build-5096 branch from 422c83e to 65d5cc9 Compare November 6, 2024 15:33

reta force-pushed the issue-build-5096 branch from 65d5cc9 to e7a48c0 Compare November 6, 2024 16:21

reta force-pushed the issue-build-5096 branch from e7a48c0 to 1774f1d Compare November 6, 2024 16:40

reta commented Nov 6, 2024

View reviewed changes

distribution/tools/plugin-cli/src/main/java/org/opensearch/plugins/InstallPluginCommand.java Show resolved Hide resolved

opensearch-ci-bot mentioned this pull request Nov 6, 2024

[AUTOCUT] Gradle Check Flaky Test Report for ResourceAwareTasksTests #14293

Open

peterzhuamazon mentioned this pull request Nov 6, 2024

[Feature] Publish plugins SNAPSHOT artifacts (along with SNASPHOT distribution) opensearch-project/opensearch-build#5096

Closed

cwperks mentioned this pull request Nov 7, 2024

Extend ./bin/opensearch-plugin install <plugin_name> to include plugins in the default distribution #16597

Closed

3 tasks

dblock approved these changes Nov 8, 2024

View reviewed changes

Support installing plugin SNAPSHOTs with SNASPHOT distribution

7b24470

Signed-off-by: Andriy Redko <[email protected]>

reta force-pushed the issue-build-5096 branch from 1255cf8 to 7b24470 Compare November 8, 2024 15:44

opensearch-ci-bot mentioned this pull request Nov 2, 2024

[AUTOCUT] Gradle Check Flaky Test Report for ShardIndexingPressureSettingsIT #14331

Open

saratvemulapalli added the Plugins label Nov 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support installing plugin SNAPSHOTs with SNASPHOT distribution #16581

Support installing plugin SNAPSHOTs with SNASPHOT distribution #16581

reta commented Nov 6, 2024 •

edited

Loading

reta Nov 6, 2024

saratvemulapalli Nov 11, 2024

reta Nov 11, 2024

reta commented Nov 6, 2024

github-actions bot commented Nov 6, 2024

github-actions bot commented Nov 6, 2024

github-actions bot commented Nov 6, 2024

github-actions bot commented Nov 6, 2024

github-actions bot commented Nov 6, 2024

codecov bot commented Nov 6, 2024 •

edited

Loading

github-actions bot commented Nov 8, 2024

github-actions bot commented Nov 8, 2024

reta commented Nov 11, 2024

reta commented Nov 12, 2024

reta commented Nov 13, 2024

Support installing plugin SNAPSHOTs with SNASPHOT distribution #16581

Are you sure you want to change the base?

Support installing plugin SNAPSHOTs with SNASPHOT distribution #16581

Conversation

reta commented Nov 6, 2024 • edited Loading

Description

Related Issues

Check List

reta Nov 6, 2024

Choose a reason for hiding this comment

saratvemulapalli Nov 11, 2024

Choose a reason for hiding this comment

reta Nov 11, 2024

Choose a reason for hiding this comment

reta commented Nov 6, 2024

github-actions bot commented Nov 6, 2024

github-actions bot commented Nov 6, 2024

github-actions bot commented Nov 6, 2024

github-actions bot commented Nov 6, 2024

github-actions bot commented Nov 6, 2024

codecov bot commented Nov 6, 2024 • edited Loading

Codecov Report

github-actions bot commented Nov 8, 2024

github-actions bot commented Nov 8, 2024

reta commented Nov 11, 2024

reta commented Nov 12, 2024

reta commented Nov 13, 2024

reta commented Nov 6, 2024 •

edited

Loading

codecov bot commented Nov 6, 2024 •

edited

Loading