CDKTF diff & deploy want to delete every resource and create identical ones with new names after upgrade from 0.16.1 to 0.17.3

[fmmoret]

Expected Behavior

Our deploys and diffs should work the way they used to or breaking changes should have documentation and upgrade instructions.

Actual Behavior

We upgraded from 0.16.1 to 0.17.3 and now every resource has a diff.
Resource names went from (as an example where my stack name is auth_staging) authstaging_nikeauthserviceappmemcacheinstance_C0A1A76C to nike_auth_service_app_memcache_instance

I think this is the cause (#2956)

Steps to Reproduce

Our environment has multiple stacks -- so synth & deploy with 0.16.1, upgrade to 0.17.3 and deploy.

Versions

~/monorepo/infra$ yarn cdktf debug
[2023-08-30T17:47:56.907] [ERROR] default - npm
npm[2023-08-30T17:47:56.911] [ERROR] default -  ERR! code ELSPROBLEMS
npm ERR! invalid: [email protected] /home/fmmoret/monorepo/infra/node_modules/cdktf

 ERR! code ELSPROBLEMS
npm ERR! invalid: [email protected] /home/fmmoret/monorepo/infra/node_modules/cdktf
[2023-08-30T17:47:56.915] [ERROR] default -


[2023-08-30T17:47:56.916] [ERROR] default - npm ERR! A complete log of this run can be found in:
npm ERR!     /home/fmmoret/.npm/_logs/2023-08-31T00_47_54_238Z-debug-0.log

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/fmmoret/.npm/_logs/2023-08-31T00_47_54_238Z-debug-0.log
cdktf debug
language: typescript
cdktf-cli: 0.17.3
node: v18.16.1
cdktf: null
constructs: 10.0.99
jsii: null
terraform: 1.5.2
arch: x64
os: linux 5.15.90.1-microsoft-standard-WSL2

cat /home/fmmoret/.npm/_logs/2023-08-31T00_47_54_238Z-debug-0.log

0 verbose cli /nix/store/l43bdmxxlkrzwic3q24ik366qyqg3s1g-nodejs-18.16.1/bin/node /nix/store/5qp126yck3fkr3mb7wbz41qvygyqb6f2-devenv-profile/bin/npm
1 info using [email protected]
2 info using [email protected]
3 timing npm:load:whichnode Completed in 1ms
4 timing config:load:defaults Completed in 2ms
5 timing config:load:file:/nix/store/l43bdmxxlkrzwic3q24ik366qyqg3s1g-nodejs-18.16.1/lib/node_modules/npm/npmrc Completed in 0ms
6 timing config:load:builtin Completed in 0ms
7 timing config:load:cli Completed in 2ms
8 timing config:load:env Completed in 1ms
9 timing config:load:file:/home/fmmoret/monorepo/infra/.npmrc Completed in 0ms
10 timing config:load:project Completed in 4ms
11 timing config:load:file:/home/fmmoret/.npmrc Completed in 0ms
12 timing config:load:user Completed in 0ms
13 timing config:load:file:/nix/store/l43bdmxxlkrzwic3q24ik366qyqg3s1g-nodejs-18.16.1/etc/npmrc Completed in 0ms
14 timing config:load:global Completed in 0ms
15 timing config:load:setEnvs Completed in 1ms
16 timing config:load Completed in 11ms
17 timing npm:load:configload Completed in 11ms
18 timing npm:load:mkdirpcache Completed in 1ms
19 timing npm:load:mkdirplogs Completed in 0ms
20 verbose title npm list constructs
21 verbose argv "list" "constructs" "--json"
22 timing npm:load:setTitle Completed in 1ms
23 timing config:load:flatten Completed in 3ms
24 timing npm:load:display Completed in 4ms
25 verbose logfile logs-max:10 dir:/home/fmmoret/.npm/_logs/2023-08-31T00_47_54_238Z-
0 verbose cli /nix/store/l43bdmxxlkrzwic3q24ik366qyqg3s1g-nodejs-18.16.1/bin/node /nix/store/5qp126yck3fkr3mb7wbz41qvygyqb6f2-devenv-profile/bin/npm
1 info using [email protected]
2 info using [email protected]
3 timing npm:load:whichnode Completed in 0ms
4 timing config:load:defaults Completed in 1ms
5 timing config:load:file:/nix/store/l43bdmxxlkrzwic3q24ik366qyqg3s1g-nodejs-18.16.1/lib/node_modules/npm/npmrc Completed in 0ms
6 timing config:load:builtin Completed in 1ms
7 timing config:load:cli Completed in 2ms
8 timing config:load:env Completed in 1ms
9 timing config:load:file:/home/fmmoret/monorepo/infra/.npmrc Completed in 0ms
10 timing config:load:project Completed in 4ms
11 timing config:load:file:/home/fmmoret/.npmrc Completed in 0ms
12 timing config:load:user Completed in 0ms
13 timing config:load:file:/nix/store/l43bdmxxlkrzwic3q24ik366qyqg3s1g-nodejs-18.16.1/etc/npmrc Completed in 0ms
14 timing config:load:global Completed in 0ms
15 timing config:load:setEnvs Completed in 1ms
16 timing config:load Completed in 11ms
17 timing npm:load:configload Completed in 11ms
18 timing npm:load:mkdirpcache Completed in 1ms
19 timing npm:load:mkdirplogs Completed in 1ms
20 verbose title npm list cdktf
21 verbose argv "list" "cdktf" "--json"
22 timing npm:load:setTitle Completed in 1ms
23 timing config:load:flatten Completed in 4ms
24 timing npm:load:display Completed in 4ms
25 verbose logfile logs-max:10 dir:/home/fmmoret/.npm/_logs/2023-08-31T00_47_54_238Z-
26 verbose logfile /home/fmmoret/.npm/_logs/2023-08-31T00_47_54_238Z-debug-0.log
27 timing npm:load:logFile Completed in 4ms
28 timing npm:load:timers Completed in 0ms
29 timing npm:load:configScope Completed in 0ms
30 timing npm:load Completed in 22ms
26 verbose logfile /home/fmmoret/.npm/_logs/2023-08-31T00_47_54_238Z-debug-0.log
27 timing npm:load:logFile Completed in 4ms
28 timing npm:load:timers Completed in 0ms
29 timing npm:load:configScope Completed in 0ms
30 timing npm:load Completed in 23ms
31 timing arborist:ctor Completed in 0ms
31 timing arborist:ctor Completed in 1ms
32 silly logfile start cleaning logs, removing 1 files
32 silly logfile start cleaning logs, removing 1 files
33 silly logfile done cleaning log files
33 silly logfile done cleaning log files
34 timing command:list Completed in 2443ms
35 verbose exit 0
36 timing npm Completed in 2480ms
37 info ok
34 timing command:list Completed in 2631ms
35 verbose stack Error: invalid: [email protected] /home/fmmoret/monorepo/infra/node_modules/cdktf
35 verbose stack     at LS.exec (/nix/store/l43bdmxxlkrzwic3q24ik366qyqg3s1g-nodejs-18.16.1/lib/node_modules/npm/lib/commands/ls.js:203:9)
35 verbose stack     at async module.exports (/nix/store/l43bdmxxlkrzwic3q24ik366qyqg3s1g-nodejs-18.16.1/lib/node_modules/npm/lib/cli.js:134:5)
36 verbose cwd /home/fmmoret/monorepo/infra
37 verbose Linux 5.15.90.1-microsoft-standard-WSL2
38 verbose node v18.16.1
39 verbose npm  v9.5.1
40 error code ELSPROBLEMS
41 error invalid: [email protected] /home/fmmoret/monorepo/infra/node_modules/cdktf
42 verbose exit 1
43 timing npm Completed in 2674ms
44 verbose code 1
45 error A complete log of this run can be found in:
45 error     /home/fmmoret/.npm/_logs/2023-08-31T00_47_54_238Z-debug-0.log

Providers

yarn cdktf provider list

[2023-08-30T17:49:46.312] [ERROR] default - npm
npm[2023-08-30T17:49:46.313] [ERROR] default -  ERR! code ELSPROBLEMS
npm ERR! invalid: [email protected] /home/fmmoret/monorepo/infra/node_modules/cdktf

 ERR! code ELSPROBLEMS
npm ERR! invalid: [email protected] /home/fmmoret/monorepo/infra/node_modules/cdktf
[2023-08-30T17:49:46.314] [ERROR] default -


[2023-08-30T17:49:46.315] [ERROR] default - npm ERR! A complete log of this run can be found in:
npm ERR!     /home/fmmoret/.npm/_logs/2023-08-31T00_49_44_976Z-debug-0.log

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/fmmoret/.npm/_logs/2023-08-31T00_49_44_976Z-debug-0.log
Error
    at Object.External (/home/fmmoret/monorepo/infra/node_modules/cdktf-cli/bundle/bin/cmds/handlers.js:94:41818)
    at Object.ORt (/home/fmmoret/monorepo/infra/node_modules/cdktf-cli/bundle/bin/cmds/handlers.js:483:34499)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async kp.handleCommand (/home/fmmoret/monorepo/infra/node_modules/cdktf-cli/bundle/bin/cdktf.js:54:885)
    at async N.handler (/home/fmmoret/monorepo/infra/node_modules/cdktf-cli/bundle/bin/cdktf.js:53:30453)
Collecting Debug Information...
/home/fmmoret/monorepo/infra/node_modules/yoga-layout-prebuilt/yoga-layout/build/Release/nbind.js:53
        throw ex;
        ^

Error
    at Object.External (/home/fmmoret/monorepo/infra/node_modules/cdktf-cli/bundle/bin/cmds/handlers.js:94:41818)
    at Object.ORt (/home/fmmoret/monorepo/infra/node_modules/cdktf-cli/bundle/bin/cmds/handlers.js:483:34499)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async kp.handleCommand (/home/fmmoret/monorepo/infra/node_modules/cdktf-cli/bundle/bin/cdktf.js:54:885)
    at async N.handler (/home/fmmoret/monorepo/infra/node_modules/cdktf-cli/bundle/bin/cdktf.js:53:30453) {
  __type: 'External'
}

cat /home/fmmoret/.npm/_logs/2023-08-31T00_49_44_976Z-debug-0.log

0 verbose cli /nix/store/l43bdmxxlkrzwic3q24ik366qyqg3s1g-nodejs-18.16.1/bin/node /nix/store/5qp126yck3fkr3mb7wbz41qvygyqb6f2-devenv-profile/bin/npm
1 info using [email protected]
2 info using [email protected]
3 timing npm:load:whichnode Completed in 1ms
4 timing config:load:defaults Completed in 1ms
5 timing config:load:file:/nix/store/l43bdmxxlkrzwic3q24ik366qyqg3s1g-nodejs-18.16.1/lib/node_modules/npm/npmrc Completed in 0ms
6 timing config:load:builtin Completed in 0ms
7 timing config:load:cli Completed in 1ms
8 timing config:load:env Completed in 0ms
9 timing config:load:file:/home/fmmoret/monorepo/infra/.npmrc Completed in 0ms
10 timing config:load:project Completed in 3ms
11 timing config:load:file:/home/fmmoret/.npmrc Completed in 0ms
12 timing config:load:user Completed in 0ms
13 timing config:load:file:/nix/store/l43bdmxxlkrzwic3q24ik366qyqg3s1g-nodejs-18.16.1/etc/npmrc Completed in 0ms
14 timing config:load:global Completed in 0ms
15 timing config:load:setEnvs Completed in 0ms
16 timing config:load Completed in 5ms
17 timing npm:load:configload Completed in 5ms
18 timing npm:load:mkdirpcache Completed in 1ms
19 timing npm:load:mkdirplogs Completed in 0ms
20 verbose title npm list cdktf
21 verbose argv "list" "cdktf" "--json"
22 timing npm:load:setTitle Completed in 1ms
23 timing config:load:flatten Completed in 2ms
24 timing npm:load:display Completed in 2ms
25 verbose logfile logs-max:10 dir:/home/fmmoret/.npm/_logs/2023-08-31T00_49_44_976Z-
26 verbose logfile /home/fmmoret/.npm/_logs/2023-08-31T00_49_44_976Z-debug-0.log
27 timing npm:load:logFile Completed in 3ms
28 timing npm:load:timers Completed in 0ms
29 timing npm:load:configScope Completed in 0ms
30 timing npm:load Completed in 13ms
31 timing arborist:ctor Completed in 0ms
32 silly logfile start cleaning logs, removing 1 files
33 silly logfile done cleaning log files
34 timing command:list Completed in 1316ms
35 verbose stack Error: invalid: [email protected] /home/fmmoret/monorepo/infra/node_modules/cdktf
35 verbose stack     at LS.exec (/nix/store/l43bdmxxlkrzwic3q24ik366qyqg3s1g-nodejs-18.16.1/lib/node_modules/npm/lib/commands/ls.js:203:9)
35 verbose stack     at async module.exports (/nix/store/l43bdmxxlkrzwic3q24ik366qyqg3s1g-nodejs-18.16.1/lib/node_modules/npm/lib/cli.js:134:5)
36 verbose cwd /home/fmmoret/monorepo/infra
37 verbose Linux 5.15.90.1-microsoft-standard-WSL2
38 verbose node v18.16.1
39 verbose npm  v9.5.1
40 error code ELSPROBLEMS
41 error invalid: [email protected] /home/fmmoret/monorepo/infra/node_modules/cdktf
42 verbose exit 1
43 timing npm Completed in 1337ms
44 verbose code 1
45 error A complete log of this run can be found in:
45 error     /home/fmmoret/.npm/_logs/2023-08-31T00_49_44_976Z-debug-0.log

Gist

No response

Possible Solutions

No response

Workarounds

No response

Anything Else?

No response

References

Help Wanted

• I'm interested in contributing a fix myself

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[fmmoret]

It appears to be a naming schema change forced instead by 17.0.

We were used to breaking changes being listed in the release descriptions here: https://github.com/hashicorp/terraform-cdk/releases
but the change is pretty low-key as a chore.

Is there an upgrade guide for the change? #2937
@DanielMSchmidt

https://github.com/hashicorp/terraform-cdk/blob/main/website/docs/cdktf/release/upgrade-guide-v0-17.mdx

CDKTF used to use feature flags to enable potentially breaking behaviors in a release. These are no longer needed since most of the changes we introduce from here can not be hidden behind feature flags. The behavior for 0.17 is the same as in 0.16 with this configuration:

```json
{
  "context": {
    "excludeStackIdFromLogicalIds": "true",
    "allowSepCharsInLogicalIds": "true"
  }
}

You can remove these keys from your `context` field in your `cdktf.json` file.

The upgrade guide w/ the release just tells me that some keys are useless now but doesn't tell me how I can fix the stack states.

[jsteinich]

#2954 lists a couple options on how to transition.

[mutahhir]

We'll add additional documentation to help migrate soon.

[fmmoret]

Was it added somewhere?

[DanielMSchmidt]

I think @mutahhir is referring to #3231. Once it's merged and release this change can be automatically done by an aspect

[mhaligowski]

I was just testing 0.19.2 with our code, and it seems it doesn't work with TerraformModule-s! It should be pretty easy to do a workaround, but it would be nice to have it in the code itself.

cc @Maed223 , author of the PR.

[ansgarm]

Hi @mhaligowski 👋

Unfortunately, we don't yet support the moved block for modules which makes it impossible to add this to the Aspect. However, I created #3432 which will print a warning with instructions with a manual terraform state mv command specific to the modules old and current id. The PR also adds a bit of docs for how to move modules until we have support for the moved block there.

[ansgarm]

Closing this as #3432 is merged now 👍

[github-actions]

I'm going to lock this issue because it has been closed for 30 days. This helps our maintainers find and focus on the active issues. If you've found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.