Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use zipfoundation for unzip #30

Merged
merged 9 commits into from
Jul 11, 2024
Merged

Conversation

finestructure
Copy link
Member

@finestructure finestructure commented Jul 10, 2024

Fixes SwiftPackageIndex/SwiftPackageIndex-Server#3137

I ended up having to go with open 2. from the above issue,

  1. use marmelroy/Zip for zipping and weichsel/ZIPFoundation for unzipping - meh, but by far the easiest option

having explored all other options, including, most recently, deploying not a Lambda binary to the provided Lambda runtime but shipping our own container with unzip installed.

However, this would be a very disruptive and risky change, because a stack cannot be updated in place from a provided runtime to a custom image runtime. The stack has to be deleted and recreated.

It would also mean having to use a container registry where we host the 600 MB images, which we need to build and push - vs the 30 MB binary zip we currently deploy.

I went through the motions of doing it and it's just too massive a change to the setup and how we deploy for fixing a single repos zip file issue, especially when there's a much simpler, albeit somewhat ugly, solution available.

@cla-bot cla-bot bot added the cla-signed label Jul 10, 2024
@finestructure
Copy link
Member Author

CI tests are still broken but I know why now: https://forums.swift.org/t/amazon-linux-2023-support/66209/10

Linux test run:

❯ docker run --rm -v "$PWD":/host -w /host swift:5.10.0-amazonlinux2 swift test

warning: 'host': dependency 'swift-crypto' is not used by any target
[0/1] Planning build
...
Build complete! (14.06s)
Test Suite 'All tests' started at 2024-07-11 08:36:04.987
Test Suite 'debug.xctest' started at 2024-07-11 08:36:04.989
Test Suite 'DocUploaderTests' started at 2024-07-11 08:36:04.989
Test Case 'DocUploaderTests.test_logUrl' started at 2024-07-11 08:36:04.989
Test Case 'DocUploaderTests.test_logUrl' passed (0.001 seconds)
Test Case 'DocUploaderTests.test_reportResult' started at 2024-07-11 08:36:04.990
Test Case 'DocUploaderTests.test_reportResult' passed (0.02 seconds)
Test Suite 'DocUploaderTests' passed at 2024-07-11 08:36:05.010
         Executed 2 tests, with 0 failures (0 unexpected) in 0.021 (0.021) seconds
Test Suite 'DocUploadBundleTests' started at 2024-07-11 08:36:05.010
Test Case 'DocUploadBundleTests.test_String_pathEncoded' started at 2024-07-11 08:36:05.010
Test Case 'DocUploadBundleTests.test_String_pathEncoded' passed (0.0 seconds)
Test Case 'DocUploadBundleTests.test_init' started at 2024-07-11 08:36:05.010
Test Case 'DocUploadBundleTests.test_init' passed (0.0 seconds)
Test Case 'DocUploadBundleTests.test_issue_10' started at 2024-07-11 08:36:05.010
Test Case 'DocUploadBundleTests.test_issue_10' passed (0.0 seconds)
Test Case 'DocUploadBundleTests.test_issue_3069' started at 2024-07-11 08:36:05.010
Test Case 'DocUploadBundleTests.test_issue_3069' passed (0.843 seconds)
Test Suite 'DocUploadBundleTests' passed at 2024-07-11 08:36:05.853
         Executed 4 tests, with 0 failures (0 unexpected) in 0.843 (0.843) seconds
Test Suite 'ZipTests' started at 2024-07-11 08:36:05.853
Test Case 'ZipTests.test_unzip' started at 2024-07-11 08:36:05.853
Test Case 'ZipTests.test_unzip' passed (0.001 seconds)
Test Case 'ZipTests.test_zip_roundtrip' started at 2024-07-11 08:36:05.854
Test Case 'ZipTests.test_zip_roundtrip' passed (0.008 seconds)
Test Suite 'ZipTests' passed at 2024-07-11 08:36:05.862
         Executed 2 tests, with 0 failures (0 unexpected) in 0.009 (0.009) seconds
Test Suite 'debug.xctest' passed at 2024-07-11 08:36:05.862
Executed 8 tests, with 0 failures (0 unexpected) in 0.873 (0.873) seconds
Test Suite 'All tests' passed at 2024-07-11 08:36:05.862
Executed 8 tests, with 0 failures (0 unexpected) in 0.873 (0.873) seconds

macOS test run:

❯ swift test
warning: 'spi-doc-uploader': dependency 'swift-crypto' is not used by any target
Building for debugging...
[59/59] Applying doc-uploader
Build complete! (18.93s)
Test Suite 'All tests' started at 2024-07-11 10:36:43.422.
Test Suite 'DocUploaderPackageTests.xctest' started at 2024-07-11 10:36:43.423.
Test Suite 'DocUploadBundleTests' started at 2024-07-11 10:36:43.423.
Test Case '-[DocUploadBundleTests.DocUploadBundleTests test_init]' started.
Test Case '-[DocUploadBundleTests.DocUploadBundleTests test_init]' passed (0.001 seconds).
Test Case '-[DocUploadBundleTests.DocUploadBundleTests test_issue_10]' started.
Test Case '-[DocUploadBundleTests.DocUploadBundleTests test_issue_10]' passed (0.000 seconds).
Test Case '-[DocUploadBundleTests.DocUploadBundleTests test_issue_3069]' started.
Test Case '-[DocUploadBundleTests.DocUploadBundleTests test_issue_3069]' passed (0.501 seconds).
Test Case '-[DocUploadBundleTests.DocUploadBundleTests test_String_pathEncoded]' started.
Test Case '-[DocUploadBundleTests.DocUploadBundleTests test_String_pathEncoded]' passed (0.000 seconds).
Test Suite 'DocUploadBundleTests' passed at 2024-07-11 10:36:43.925.
         Executed 4 tests, with 0 failures (0 unexpected) in 0.502 (0.502) seconds
Test Suite 'DocUploaderTests' started at 2024-07-11 10:36:43.925.
Test Case '-[DocUploaderTests.DocUploaderTests test_logUrl]' started.
Test Case '-[DocUploaderTests.DocUploaderTests test_logUrl]' passed (0.000 seconds).
Test Case '-[DocUploaderTests.DocUploaderTests test_reportResult]' started.
Test Case '-[DocUploaderTests.DocUploaderTests test_reportResult]' passed (0.002 seconds).
Test Suite 'DocUploaderTests' passed at 2024-07-11 10:36:43.928.
         Executed 2 tests, with 0 failures (0 unexpected) in 0.003 (0.003) seconds
Test Suite 'ZipTests' started at 2024-07-11 10:36:43.928.
Test Case '-[DocUploadBundleTests.ZipTests test_unzip]' started.
Test Case '-[DocUploadBundleTests.ZipTests test_unzip]' passed (0.001 seconds).
Test Case '-[DocUploadBundleTests.ZipTests test_zip_roundtrip]' started.
Test Case '-[DocUploadBundleTests.ZipTests test_zip_roundtrip]' passed (0.002 seconds).
Test Suite 'ZipTests' passed at 2024-07-11 10:36:43.932.
         Executed 2 tests, with 0 failures (0 unexpected) in 0.004 (0.004) seconds
Test Suite 'DocUploaderPackageTests.xctest' passed at 2024-07-11 10:36:43.932.
         Executed 8 tests, with 0 failures (0 unexpected) in 0.509 (0.509) seconds
Test Suite 'All tests' passed at 2024-07-11 10:36:43.932.
         Executed 8 tests, with 0 failures (0 unexpected) in 0.509 (0.510) seconds

@finestructure finestructure marked this pull request as ready for review July 11, 2024 09:01
@finestructure finestructure merged commit 4887d10 into main Jul 11, 2024
2 checks passed
@finestructure
Copy link
Member Author

Will deploy this and test swift-metrics doc gen next week after we're done with Xcode 16b3 processing.

@finestructure finestructure deleted the use-zipfoundation-for-unzip branch July 11, 2024 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Properly fix doc generation issue
2 participants