You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We can't remove and then redeploy any optional stacks. When you remove an optional stack, the log groups are retained. When you try to add it back in, it fails to deploy because the log groups already exist.
Steps to reproduce
Deploy an instance with the optional stack 'PersistentEMRBulkImportStack'
Use the admin client to remove that stack, and wait for the CDK to run to remove the cluster
Run the admin client again to add that stack back in
See it fails because it attempts to create a log group for the persistent cluster, but the log group already exists.
Expected behaviour
It should be possible to deploy, remove, redeploy, ... the PersistentEMRBulkImportStack.
Background
We have this issue which suggests making the persistent cluster a runtime deployment option, similar to the non-persistent EMR stack. We may choose to do that and that might solve this issue.
We probably have the same issue for all of the optional stacks. We may be able to solve this by always deploying log groups for all optional stacks and then pointing those stacks at those log groups when they are deployed.
We've split out a separate issue to resolve this for bulk import on EKS:
This seems like a more general problem with optional stacks. I think most if not all of the optional stacks include a log group that would cause this problem when it's removed and added back in.
gaffer01
changed the title
Can't redeploy persistent EMR cluster
Can't redeploy persistent EMR cluster or other optional stacks
Sep 13, 2024
Description
We can't remove and then redeploy any optional stacks. When you remove an optional stack, the log groups are retained. When you try to add it back in, it fails to deploy because the log groups already exist.
Steps to reproduce
Expected behaviour
It should be possible to deploy, remove, redeploy, ... the PersistentEMRBulkImportStack.
Background
We have this issue which suggests making the persistent cluster a runtime deployment option, similar to the non-persistent EMR stack. We may choose to do that and that might solve this issue.
We probably have the same issue for all of the optional stacks. We may be able to solve this by always deploying log groups for all optional stacks and then pointing those stacks at those log groups when they are deployed.
We've split out a separate issue to resolve this for bulk import on EKS:
The text was updated successfully, but these errors were encountered: