Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

amplify build is breaking backend override #11791

Closed
2 tasks done
JefStat opened this issue Jan 17, 2023 · 17 comments
Closed
2 tasks done

amplify build is breaking backend override #11791

JefStat opened this issue Jan 17, 2023 · 17 comments
Assignees
Labels
pending-triage Issue is pending triage platform Issues tied to the general CLI platform

Comments

@JefStat
Copy link

JefStat commented Jan 17, 2023

How did you install the Amplify CLI?

npm

If applicable, what version of Node.js are you using?

16.18.1

Amplify CLI Version

10.6.1

What operating system are you using?

Mac

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

No manual changes made

Describe the bug

It's been 2 months since I last deployed or touched my amplify project. Getting back to it today to add analytics I couldn't deploy due to errors.

I updated the amplify cli to latest but I'm not sure what version I updated from.

Expected behavior

I expect amplify build not to break subsequent builds

Reproduction steps

  1. amplify checkout dev --restore
  2. amplify build (no issues, but lots of changes in backend-config.json and amplify-dependent-resources-ref.d.ts. Mostly looks like properties are now in alphabetical order)
  3. amplify build (second time)
    error Failed to override pccapi due to: InvalidOverrideError: Packaging overrides failed..

Project Identifier

729269ed0dfa7238ed8a5b3b8f9c5f9a

Log output

2023-01-16T16:58:57.492Z|info : amplify env core checkout dev {"restore":true,"yes":false}
2023-01-16T16:58:58.340Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileConfig(["XXXX"])
2023-01-16T16:58:58.340Z|info : amplify-provider-awscloudformation.system-config-manager.getProfiledAwsConfig.profileConfig([{"region":"us-west-2"}])
2023-01-16T16:58:58.340Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileCredentials(["XXXX"])
2023-01-16T16:58:58.346Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileConfig(["XXXX"])
2023-01-16T16:58:58.347Z|info : amplify-provider-awscloudformation.system-config-manager.getProfiledAwsConfig.profileConfig([{"region":"us-west-2"}])
2023-01-16T16:58:58.347Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileCredentials(["XXXX"])
2023-01-16T16:58:58.351Z|info : amplify-provider-awscloudformation.zip-util.downloadZip.s3.getFile([{"Key":"[***]rent-[***]ud-[***]d.zip"},null])
2023-01-16T16:58:58.351Z|info : amplify-provider-awscloudformation.aws-s3.s3.getFile([{"Key":"[***]rent-[***]ud-[***]d.zip","Bucket":"[***]ify-[***]dcentral-[***]ev-[***]522-[***]ment"}])
2023-01-16T16:59:01.764Z|info : amplify-provider-awscloudformation.initialize-env.run.cfn.updateamplifyMetaFileWithStackOutputs([{"StackName":"[***]ify-[***]dcentral-[***]ev-[***]522"}])
2023-01-16T16:59:01.765Z|info : amplify-provider-awscloudformation.aws-cfn.updateamplifyMetaFileWithStackOutputs.cfn.listStackResources([{"StackName":"[***]ify-[***]dcentral-[***]ev-[***]522"}])
2023-01-16T16:59:02.574Z|info : amplify-provider-awscloudformation.aws-cfn.describeStack.cfn.describeStacks([{"StackName":"[***]ify-[***]dcentral-[***]ev-[***]522"}])
2023-01-16T16:59:03.029Z|info : amplify-provider-awscloudformation.aws-cfn.describeStack.cfn.describeStacks([{"StackName":"[***]rmation:us-[***]st-[***]92:stack/[***]ify-[***]dcentral-[***]ev-[***]522-[***]uthStack-[***]WTE547/[***]2e10-[***]41-[***]ed-[***]9f-[***]47b19"}])
2023-01-16T16:59:03.030Z|info : amplify-provider-awscloudformation.aws-cfn.describeStack.cfn.describeStacks([{"StackName":"[***]rmation:us-[***]st-[***]92:stack/[***]ify-[***]dcentral-[***]ev-[***]522-[***]capi-[***]0A70NA/[***]6ac0-[***]41-[***]ed-[***]db-[***]2dbe7"}])
2023-01-16T16:59:03.031Z|info : amplify-provider-awscloudformation.aws-cfn.describeStack.cfn.describeStacks([{"StackName":"[***]rmation:us-[***]st-[***]92:stack/[***]ify-[***]dcentral-[***]ev-[***]522-[***]ntral772995b7-[***]YF4CY/[***]ea10-[***]1b-[***]ed-[***]00-[***]c8217"}])
2023-01-16T16:59:03.033Z|info : amplify-provider-awscloudformation.aws-cfn.describeStack.cfn.describeStacks([{"StackName":"[***]rmation:us-[***]st-[***]92:stack/[***]ify-[***]dcentral-[***]ev-[***]522-[***]olGroups-[***]Y7P9SD/[***]c010-[***]c0-[***]ed-[***]08-[***]db3e1"}])
2023-01-16T16:59:03.034Z|info : amplify-provider-awscloudformation.aws-cfn.describeStack.cfn.describeStacks([{"StackName":"[***]rmation:us-[***]st-[***]92:stack/[***]ify-[***]dcentral-[***]ev-[***]522-[***]erDbCRUD-[***]H39CKA/[***]c650-[***]4a-[***]ed-[***]2d-[***]9b319"}])
2023-01-16T16:59:03.035Z|info : amplify-provider-awscloudformation.aws-cfn.describeStack.cfn.describeStacks([{"StackName":"[***]rmation:us-[***]st-[***]92:stack/[***]ify-[***]dcentral-[***]ev-[***]522-[***]rconfigs-[***]7PO90/[***]5420-[***]4a-[***]ed-[***]e4-[***]5fe15"}])
2023-01-16T16:59:03.704Z|info : amplify-provider-awscloudformation.aws-s3.s3.getFile([{"Key":"[***]ify-[***]json","Bucket":"[***]ify-[***]dcentral-[***]ev-[***]522-[***]ment"}])
2023-01-16T18:13:57.774Z|info : amplify add analytics  
2023-01-16T18:13:57.809Z|info : amplify-cli-core.banner-message/index.ts.fetch banner messages from https://aws-amplify.github.io/amplify-cli/banner-message.json({}
2023-01-16T18:15:21.374Z|info : amplify upgrade core  
2023-01-16T18:15:21.415Z|info : amplify-cli-core.banner-message/index.ts.fetch banner messages from https://aws-amplify.github.io/amplify-cli/banner-message.json({}
2023-01-16T18:17:29.099Z|info : amplify push core  
2023-01-16T18:17:29.147Z|info : amplify-cli-core.banner-message/index.ts.fetch banner messages from https://aws-amplify.github.io/amplify-cli/banner-message.json({}
2023-01-16T18:17:30.241Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileConfig(["XXXX"])
2023-01-16T18:17:30.241Z|info : amplify-provider-awscloudformation.system-config-manager.getProfiledAwsConfig.profileConfig([{"region":"us-west-2"}])
2023-01-16T18:17:30.242Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileCredentials(["XXXX"])
2023-01-16T18:17:30.247Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileConfig(["XXXX"])
2023-01-16T18:17:30.248Z|info : amplify-provider-awscloudformation.system-config-manager.getProfiledAwsConfig.profileConfig([{"region":"us-west-2"}])
2023-01-16T18:17:30.248Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileCredentials(["XXXX"])
2023-01-16T18:17:30.252Z|info : amplify-provider-awscloudformation.zip-util.downloadZip.s3.getFile([{"Key":"[***]rent-[***]ud-[***]d.zip"},null])
2023-01-16T18:17:30.252Z|info : amplify-provider-awscloudformation.aws-s3.s3.getFile([{"Key":"[***]rent-[***]ud-[***]d.zip","Bucket":"[***]ify-[***]dcentral-[***]ev-[***]522-[***]ment"}])
2023-01-16T18:17:32.411Z|info : amplify-provider-awscloudformation.initialize-env.run.cfn.updateamplifyMetaFileWithStackOutputs([{"StackName":"[***]ify-[***]dcentral-[***]ev-[***]522"}])
2023-01-16T18:17:32.413Z|info : amplify-provider-awscloudformation.aws-cfn.updateamplifyMetaFileWithStackOutputs.cfn.listStackResources([{"StackName":"[***]ify-[***]dcentral-[***]ev-[***]522"}])
2023-01-16T18:17:33.041Z|info : amplify-provider-awscloudformation.aws-cfn.describeStack.cfn.describeStacks([{"StackName":"[***]ify-[***]dcentral-[***]ev-[***]522"}])
2023-01-16T18:17:33.658Z|info : amplify-provider-awscloudformation.aws-cfn.describeStack.cfn.describeStacks([{"StackName":"[***]rmation:us-[***]st-[***]92:stack/[***]ify-[***]dcentral-[***]ev-[***]522-[***]uthStack-[***]WTE547/[***]2e10-[***]41-[***]ed-[***]9f-[***]47b19"}])
2023-01-16T18:17:33.659Z|info : amplify-provider-awscloudformation.aws-cfn.describeStack.cfn.describeStacks([{"StackName":"[***]rmation:us-[***]st-[***]92:stack/[***]ify-[***]dcentral-[***]ev-[***]522-[***]capi-[***]0A70NA/[***]6ac0-[***]41-[***]ed-[***]db-[***]2dbe7"}])
2023-01-16T18:17:33.661Z|info : amplify-provider-awscloudformation.aws-cfn.describeStack.cfn.describeStacks([{"StackName":"[***]rmation:us-[***]st-[***]92:stack/[***]ify-[***]dcentral-[***]ev-[***]522-[***]ntral772995b7-[***]YF4CY/[***]ea10-[***]1b-[***]ed-[***]00-[***]c8217"}])
2023-01-16T18:17:33.662Z|info : amplify-provider-awscloudformation.aws-cfn.describeStack.cfn.describeStacks([{"StackName":"[***]rmation:us-[***]st-[***]92:stack/[***]ify-[***]dcentral-[***]ev-[***]522-[***]olGroups-[***]Y7P9SD/[***]c010-[***]c0-[***]ed-[***]08-[***]db3e1"}])
2023-01-16T18:17:33.663Z|info : amplify-provider-awscloudformation.aws-cfn.describeStack.cfn.describeStacks([{"StackName":"[***]rmation:us-[***]st-[***]92:stack/[***]ify-[***]dcentral-[***]ev-[***]522-[***]erDbCRUD-[***]H39CKA/[***]c650-[***]4a-[***]ed-[***]2d-[***]9b319"}])
2023-01-16T18:17:33.665Z|info : amplify-provider-awscloudformation.aws-cfn.describeStack.cfn.describeStacks([{"StackName":"[***]rmation:us-[***]st-[***]92:stack/[***]ify-[***]dcentral-[***]ev-[***]522-[***]rconfigs-[***]7PO90/[***]5420-[***]4a-[***]ed-[***]e4-[***]5fe15"}])
2023-01-16T18:17:34.211Z|info : amplify-provider-awscloudformation.aws-s3.s3.getFile([{"Key":"[***]ify-[***]json","Bucket":"[***]ify-[***]dcentral-[***]ev-[***]522-[***]ment"}])
2023-01-16T18:18:08.706Z|error : Packaging lambda function failed with the error 
Command failed with exit code 127: npm run-script amplify:parserDbCRUD
/var/folders/sg/vcm3djgs71zbj9wyznxgtvbw0000gn/T/amplifyparserDbCRUD-800ebd8e.sh: line 1: node_modules/.bin/tsc: No such file or directory

> [email protected] amplify:XXXX
> cd amplify/backend/function/XXXX/src && npm i && node_modules/.bin/tsc -p ./tsconfig.json && cd -


changed 3 packages, and audited 293 packages in 778ms

10 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
DeploymentFault: Packaging lambda function failed with the error 
Command failed with exit code 127: npm run-script amplify:XXXX
/var/folders/sg/vcm3djgs71zbj9wyznxgtvbw0000gn/T/amplifyparserXXXX-800ebd8e.sh: line 1: node_modules/.bin/tsc: No such file or directory

> [email protected] amplify:XXXX
> cd amplify/backend/function/XXXX/src && npm i && node_modules/.bin/tsc -p ./tsconfig.json && cd -


changed 3 packages, and audited 293 packages in 778ms

10 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities


Additional information

No response

Before submitting, please confirm:

  • I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
  • I have removed any sensitive information from my code snippets and submission.
@JefStat JefStat added the pending-triage Issue is pending triage label Jan 17, 2023
@josefaidt
Copy link
Contributor

Hey @JefStat 👋 thanks for raising this! As we begin to look into this a bit further would you mind sharing the affected override for pccapi?

@josefaidt josefaidt added platform Issues tied to the general CLI platform pending-response Issue is pending response from the issue author labels Jan 17, 2023
@JefStat
Copy link
Author

JefStat commented Jan 17, 2023

sure just the override ts?

override.ts

// This file is used to override the REST API resources configuration
import { AmplifyApiRestResourceStackTemplate } from '@aws-amplify/cli-extensibility-helper';

export function override(resources: AmplifyApiRestResourceStackTemplate) {
  // Add a parameter to your Cloud Formation Template for the User Pool's ID
  resources.addCfnParameter({
      type: "String",
      description: "The id of an existing User Pool to connect. If this is changed, a user pool will not be created for you.",
      default: "NONE",
    },
    "AuthCognitoUserPoolId",
    { "Fn::GetAtt": ["authl772995b7", "Outputs.UserPoolId"], }
  );

  // Create the authorizer using the AuthCognitoUserPoolId parameter defined above
  resources.restApi.addPropertyOverride("Body.securityDefinitions", {
    Cognito: {
      type: "apiKey",
      name: "Authorization",
      in: "header",
      "x-amazon-apigateway-authtype": "cognito_user_pools",
      "x-amazon-apigateway-authorizer": {
        type: "cognito_user_pools",
        providerARNs: [
          { 'Fn::Sub': 'arn:aws:cognito-idp:${AWS::Region}:${AWS::AccountId}:userpool/${AuthCognitoUserPoolId}' },
        ],
      },
    },
  });

  // For every path in our REST API
  for (const path in resources.restApi.body.paths) {
    // Add the Authorization header as a parameter to requests
    resources.restApi.addPropertyOverride(
      `Body.paths.${path}.x-amazon-apigateway-any-method.parameters`,
      [
        ...resources.restApi.body.paths[path]["x-amazon-apigateway-any-method"]
          .parameters,
        {
          name: "Authorization",
          in: "header",
          required: false,
          type: "string",
        },
      ]
    );
    // Use our new Cognito User Pool authorizer for security
    resources.restApi.addPropertyOverride(
      `Body.paths.${path}.x-amazon-apigateway-any-method.security`,
      [ { Cognito: [], }, ]
    );
  }

  resources.restApi.binaryMediaTypes = ["application~1octet-stream", "application~1zip"];
}

@github-actions github-actions bot removed the pending-response Issue is pending response from the issue author label Jan 17, 2023
@JefStat
Copy link
Author

JefStat commented Jan 17, 2023

Running amplify build from source dev branch generates the following log

/node /Users/xxx/projects/amplify-cli/packages/amplify-cli/bin/amplify build
(node:57013) [DEP0128] DeprecationWarning: Invalid 'main' field in '/Users/xxx/projects/amplify-cli/node_modules/cloudform/package.json' of 'packages/cloudform/index.js'. Please either fix that or report it to the module author
(Use `node --trace-deprecation ...` to show where the warning was created)

@josefaidt
Copy link
Contributor

Hey @JefStat thanks for posting that! While I don't see anything that immediately stands out in the posted override code, can you try re-running that command with the --debug flag?

@josefaidt josefaidt added the pending-response Issue is pending response from the issue author label Jan 17, 2023
@JefStat
Copy link
Author

JefStat commented Jan 18, 2023

First time running build nothing the second time

amplify build --debug
Overrides functionality is not implemented for this category
amplify push --debug
⠦ Fetching updates to backend environment: prod from the cloud.Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
✔ Successfully pulled backend environment prod from the cloud.
Overrides functionality is not implemented for this category

@github-actions github-actions bot removed the pending-response Issue is pending response from the issue author label Jan 18, 2023
@Ph222222
Copy link

I have the exact same issue. Can you please advise regarding the resolution timeline?

@acusti
Copy link

acusti commented Jan 25, 2023

i am also seeing this same issue. my override.ts is being used to add a Cognito User Pool authorizer to my REST API (as described in the docs). i am using v10.6.2 of the amplify CLI, and amplify push --debug results in:

⠋ Fetching updates to backend environment: main from the cloud.Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
🛑 Failed to override publicAPI due to: InvalidOverrideError: Packaging overrides failed..

@JefStat
Copy link
Author

JefStat commented Jan 25, 2023

This won't be very helpful but after trying to debug this issue by running it from the amplify cli source it resolved itself. Supposing the issue is a problem with the host machine I'm using, another change was I had recently updated OSX to Ventura.

@acusti
Copy link

acusti commented Jan 25, 2023

mmm, i also recently update to Ventura (macOS 13.1).

@josefaidt josefaidt self-assigned this Jan 25, 2023
@josefaidt
Copy link
Contributor

Hey @JefStat and @acusti I was able to reproduce an issue if my override is attempting to reference a resource

resources.addCfnParameter(
    {
      type: 'String',
      description:
        'The id of an existing User Pool to connect. If this is changed, a user pool will not be created for you.',
      default: 'NONE',
    },
    'AuthCognitoUserPoolId',
    { 'Fn::GetAtt': ['117236f41090f', 'Outputs.UserPoolId'] }
  )

Here, although my resource is named 117236f41090f I had to add the "auth" prefix as shown in @JefStat's example to proceed

auth117236f41090f

When this override fails to package the push attempt flashes with an error and quickly proceeds to the CFN update without actually kicking off the CFN update. This causes the CLI to hang indefinitely and requires manually cancelling out of the flow (as described in #11607)

Can you verify the resource references use the correct resource name, and confirm whether auth also has an override applied?

Despite this, I am not able to reproduce the issue where subsequent amplify build attempts break the packaging. Since amplify build appears to work when executed and the overrides fail to package on amplify push this leads me to believe there is an incorrect reference or some other issue.

@josefaidt josefaidt added the pending-response Issue is pending response from the issue author label Jan 25, 2023
@acusti
Copy link

acusti commented Jan 26, 2023

the auth prefix is also in the docs ({ "Fn::GetAtt": ["auth<your auth name here>", "Outputs.UserPoolId"], }), and my override.ts includes it. also, the resource reference is using the correct resource name (which is the name of the folder inside amplify/backend/auth/). i haven’t actually made any changes to the override.ts file since the beginning of june and it had been working fine until recently. unfortunately, i can’t say when it stopped working, because it’s been a while since i last tried to run amplify push, but it’s definitely broken now.

and confirm whether auth also has an override applied?

i do not have any overrides applied to auth. the REST API override is the only one in my entire application.

@Ph222222
Copy link

The issue is resolved after upgrading MacOS Ventura to 13.2 (22D49)

@josefaidt
Copy link
Contributor

Hey folks apologies for the delay here but I wanted to follow up. Are you still experiencing this with the latest version of Amplify CLI (10.7.3)? Are there any custom resources in the project that are depending on the amplify-dependent-resources.d.ts file? There is an active bug where this can cause amplify build to fail, but typically the messaging will be different. Unfortunately I have not been able to reproduce this issue

@josefaidt
Copy link
Contributor

Hey @JefStat and folks in the thread, are you still experiencing this issue?

@JefStat
Copy link
Author

JefStat commented Mar 21, 2023

I'm past this issue ty.

@github-actions github-actions bot removed the pending-response Issue is pending response from the issue author label Mar 21, 2023
@github-actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

@acusti
Copy link

acusti commented Mar 21, 2023

@josefaidt in my app, my real issue was #11851, so i’m not having issues anymore since upgrading the CLI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pending-triage Issue is pending triage platform Issues tied to the general CLI platform
Projects
None yet
Development

No branches or pull requests

4 participants