Use new version of deb-s3 for validating job publishing #16305
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
!ci-build-me |
|
!ci-build-me |
|
Archive unit tests are failing all the time after updating toolchain: https://buildkite.com/o-1-labs-2/mina-o-1-labs/builds/35968#0192d971-40d3-42ad-8114-4307f749752a due to yet again using different port. I need to fix it in this PR not to introduce regression |
|
Small update. I created official release for deb-s3 in version 0.11.7 . I will now update PR. CI will use official version |
|
!ci-build-me |
deepthiskumar
approved these changes
Nov 7, 2024
SanabriaRusso
approved these changes
Nov 11, 2024
This was referenced Nov 11, 2024
|
!ci-build-me |
|
!approved-for-mainnet |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Used new deb-s3 version with new command (deb-s3 exist) which is capable of verifying existence of more than one packages in Packages file for given codename and component.
@mrmr1993 rightly pointed out that majority of publish packages verification job duration is spent on downloading Packages file (which can take ~ 45 seconds, since file can be up to 60 Mb) for each package. New deb-s3 version is capable to run validation for multiple debs in one go. We are utilizing this new command in publish.sh script.
Output of deb-s3 exist command is multiline tabular string in below format:
We are parsing such output and learning what packages are still missing. Then rerun command for only remaining ones.
We can now observe that publish job duration decreased from ~10+ minutes to only 2 minutes.
Note for reviewers:
I'm using 0.11.7 alpha version for deb-s3, since i didn't want to merge code to master of minaprotocol/deb-s3 repo . Therefore i'm asking to first review this PR along with: MinaProtocol/deb-s3#1
Then I will update this PR to use 0.11.7 version