Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

s3_object - allow recursive copy of all objects in S3 bucket #1608

Merged

Conversation

abikouo
Copy link
Contributor

@abikouo abikouo commented Jun 12, 2023

SUMMARY

Add support to copy recursively all objects from one bucket to another one, user can set prefix to limit the object to copy.
closes #1379

ISSUE TYPE
  • Feature Pull Request
COMPONENT NAME

s3_object

@github-actions
Copy link

github-actions bot commented Jun 12, 2023

Docs Build 📝

Thank you for contribution!✨

This PR has been merged and your docs changes will be incorporated when they are next published.

@softwarefactory-project-zuul
Copy link
Contributor

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/cd4aa3f2765e40f292c24f87c2039409

✔️ ansible-galaxy-importer SUCCESS in 4m 05s
✔️ build-ansible-collection SUCCESS in 12m 51s
✔️ ansible-test-splitter SUCCESS in 5m 14s
integration-amazon.aws-1 FAILURE in 7m 48s
Skipped 43 jobs

@softwarefactory-project-zuul
Copy link
Contributor

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/91dfc696fd0a41cc862ef887be915868

✔️ ansible-galaxy-importer SUCCESS in 3m 48s
✔️ build-ansible-collection SUCCESS in 13m 39s
✔️ ansible-test-splitter SUCCESS in 4m 44s
integration-amazon.aws-1 FAILURE in 13m 32s
Skipped 43 jobs

@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/5f013ec7bfa74786a75c7617d6fdad56

✔️ ansible-galaxy-importer SUCCESS in 4m 07s
✔️ build-ansible-collection SUCCESS in 12m 57s
✔️ ansible-test-splitter SUCCESS in 5m 19s
✔️ integration-amazon.aws-1 SUCCESS in 9m 52s
Skipped 43 jobs

@hakbailey hakbailey self-requested a review June 27, 2023 13:33
plugins/modules/s3_object.py Outdated Show resolved Hide resolved
plugins/modules/s3_object.py Show resolved Hide resolved
plugins/modules/s3_object.py Outdated Show resolved Hide resolved
@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/09edc6d1b7ad44d6b85bb994b1589b98

✔️ ansible-galaxy-importer SUCCESS in 4m 36s
✔️ build-ansible-collection SUCCESS in 13m 01s
✔️ ansible-test-splitter SUCCESS in 5m 38s
✔️ integration-amazon.aws-1 SUCCESS in 11m 58s
Skipped 43 jobs

@abikouo abikouo requested a review from hakbailey June 28, 2023 07:40
@abikouo abikouo added the mergeit Merge the PR (SoftwareFactory) label Jul 3, 2023
@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded (gate pipeline).
https://ansible.softwarefactory-project.io/zuul/buildset/81fd444c0eff40928320b141d44afd2b

✔️ ansible-galaxy-importer SUCCESS in 5m 26s
✔️ build-ansible-collection SUCCESS in 12m 58s
✔️ ansible-test-splitter SUCCESS in 4m 53s
✔️ integration-amazon.aws-1 SUCCESS in 22m 07s
Skipped 43 jobs

@softwarefactory-project-zuul softwarefactory-project-zuul bot merged commit 3594dec into ansible-collections:main Jul 3, 2023
abikouo pushed a commit to abikouo/amazon.aws that referenced this pull request Sep 18, 2023
…ions against existing stacksets complete (ansible-collections#1790)

cloudformation_stack_set - Add a waiter to ensure that running operations against existing stacksets complete

SUMMARY
Add a waiter to ensure that running operations against existing stacksets complete. Current code would fail in cases where new instances need to be added since the previous update_stack_set(module, stack_params, cfn) would still be running.
Fixes ansible-collections#1608
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
cloudformation_stack_set
ADDITIONAL INFORMATION
I initially thought that the module was not idempotent since new instances wouldn't be added to my existing stack sets. Upon closer examination, the issue had to do with the fact that we had prior calls being made before adding new instances to existing stack sets:
raise error_class(parsed_response, operation_name)\nbotocore.errorfactory.OperationInProgressException: 
    An error occurred (OperationInProgressException) when calling the UpdateStackInstances operation:
    Another Operation on StackSet arn:aws:cloudformation:us-east-1:XXXXXX:stackset/aws-config-stackset:2bcb419a-f263-48ca-9fe0-cdef11fb59de is in progress
The error got triggered because of a missing waiter after this operation:
changed |= update_stack_set(module, stack_params, cfn)
This change add a waiter function after the update operation, which, in turn, ensure that the subsequent call to add stack instances to the stack set properly run.

Reviewed-by: Mark Chappell
abikouo pushed a commit to abikouo/amazon.aws that referenced this pull request Sep 18, 2023
…ions against existing stacksets complete (ansible-collections#1790)

cloudformation_stack_set - Add a waiter to ensure that running operations against existing stacksets complete

SUMMARY
Add a waiter to ensure that running operations against existing stacksets complete. Current code would fail in cases where new instances need to be added since the previous update_stack_set(module, stack_params, cfn) would still be running.
Fixes ansible-collections#1608
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
cloudformation_stack_set
ADDITIONAL INFORMATION
I initially thought that the module was not idempotent since new instances wouldn't be added to my existing stack sets. Upon closer examination, the issue had to do with the fact that we had prior calls being made before adding new instances to existing stack sets:
raise error_class(parsed_response, operation_name)\nbotocore.errorfactory.OperationInProgressException: 
    An error occurred (OperationInProgressException) when calling the UpdateStackInstances operation:
    Another Operation on StackSet arn:aws:cloudformation:us-east-1:XXXXXX:stackset/aws-config-stackset:2bcb419a-f263-48ca-9fe0-cdef11fb59de is in progress
The error got triggered because of a missing waiter after this operation:
changed |= update_stack_set(module, stack_params, cfn)
This change add a waiter function after the update operation, which, in turn, ensure that the subsequent call to add stack instances to the stack set properly run.

Reviewed-by: Mark Chappell
abikouo pushed a commit to abikouo/amazon.aws that referenced this pull request Oct 24, 2023
…ions against existing stacksets complete (ansible-collections#1790)

cloudformation_stack_set - Add a waiter to ensure that running operations against existing stacksets complete

SUMMARY
Add a waiter to ensure that running operations against existing stacksets complete. Current code would fail in cases where new instances need to be added since the previous update_stack_set(module, stack_params, cfn) would still be running.
Fixes ansible-collections#1608
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
cloudformation_stack_set
ADDITIONAL INFORMATION
I initially thought that the module was not idempotent since new instances wouldn't be added to my existing stack sets. Upon closer examination, the issue had to do with the fact that we had prior calls being made before adding new instances to existing stack sets:
raise error_class(parsed_response, operation_name)\nbotocore.errorfactory.OperationInProgressException: 
    An error occurred (OperationInProgressException) when calling the UpdateStackInstances operation:
    Another Operation on StackSet arn:aws:cloudformation:us-east-1:XXXXXX:stackset/aws-config-stackset:2bcb419a-f263-48ca-9fe0-cdef11fb59de is in progress
The error got triggered because of a missing waiter after this operation:
changed |= update_stack_set(module, stack_params, cfn)
This change add a waiter function after the update operation, which, in turn, ensure that the subsequent call to add stack instances to the stack set properly run.

Reviewed-by: Mark Chappell
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
mergeit Merge the PR (SoftwareFactory)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow recursive copy of objects in S3 bucket
2 participants