Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Abort/Cancel uploads not working, initiated by S3TransferManager #5605

Open
1 task
ramakrishna-g1 opened this issue Sep 18, 2024 · 0 comments
Open
1 task
Labels
bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.

Comments

@ramakrishna-g1
Copy link

Describe the bug

We have triggered uploads for 2k files.
Multiple files uploads triggered using S3TransferManager.
transferManager.uploadFile(uploadFileRequest).
Uploads are not cancelled/stopped when transferManager.close(); used in case of any exception while uploads in progress.

Regression Issue

  • Select this option if this issue appears to be a regression.

Expected Behavior

Should be able to abort ongoing S3 uploads even after closing transfer manager using transferManager.close();

Current Behavior

S3 Uploads are continuing after transferManager.close();

Reproduction Steps

Below code snippet is used for creating S3TransferManager &
S3AsyncClient.
----- create S3AsyncClient ------
public S3AsyncClient getS3AsyncClient(ThreadPoolExecutor threadPoolExecutor) {

	long MB = 1024L * 1024L;        
    S3CrtHttpConfiguration s3CrtHttpConfiguration = S3CrtHttpConfiguration.builder()
            .connectionTimeout(Duration.of(30, ChronoUnit.SECONDS))
            .build();

    return S3AsyncClient.crtBuilder()
            .region(Region.of("us-west-2"))
            .credentialsProvider(credentials)
            .minimumPartSizeInBytes(30 * MB)
            .httpConfiguration(s3CrtHttpConfiguration)
            .targetThroughputInGbps(30d)
            .maxConcurrency(50)
            .futureCompletionExecutor(threadPoolExecutor)
            .build();
}

----- create transfer manager ------
public S3TransferManager createTransferManager(S3AsyncClient s3AsyncClient, ThreadPoolExecutor threadPoolExecutor) {
return S3TransferManager.builder()
.s3Client(s3AsyncClient)
.executor(threadPoolExecutor)
.build();
}
----- Upload logic ------
List fileUploadList=new ArrayList(0);

	for (int i = 0; i < 1000; i++) {
		File file = new File("");
		 UploadFileRequest uploadFileRequest =
				UploadFileRequest.builder()
						.addTransferListener(LoggingTransferListener.create())
						.putObjectRequest(putRequest -> {
							putRequest.bucket("dev-bucket")
									.key("/test/"+file.getName())
									.build();
						})
						.source(file)
						.build();

		 fileUploadList.add(transferManager.uploadFile(uploadFileRequest));
	}

Abort logic -
uploads are not stopped / aborted in case of any issues/exception when
transferManager.close(); is used.

Possible Solution

SDK should have mechanism to abort ongoing uploads.

Additional Information/Context

No response

AWS Java SDK version used

2.25.65

JDK version used

17

Operating System and version

Ubuntu 20.04.6 LTS

@ramakrishna-g1 ramakrishna-g1 added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Sep 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.
Projects
None yet
Development

No branches or pull requests

1 participant