Shopping List tutorial: hitting build failure with get-shopping-list.ts

[Analect]

I was working through the Shopping List tutorial but have hit a problem at the end of Submodule 3 when it comes to run pdk build. The build is failing on root-directory/packages/api/handlers/typescript/src/get-shopping-lists.ts. I have been following the tutorial closely, but it appears certain syntax in this file introduced in 'Implement the Handlers' at the end of Submodule 2 is causing a problem. Any suggestions on how I might fix this? Thanks.

myapi-typescript-handlers: src/get-shopping-lists.ts(24,46): error TS2351: This expression is not constructable.
myapi-typescript-handlers:   Each member of the union type 'typeof QueryCommand | typeof ScanCommand' has construct signatures, but none of those signatures are compatible with each other.
myapi-typescript-handlers: src/get-shopping-lists.ts(40,24): error TS2347: Untyped function calls may not accept type arguments.
myapi-typescript-handlers: src/get-shopping-lists.ts(40,34): error TS2339: Property 'Items' does not exist on type 'ServiceOutputTypes'.
myapi-typescript-handlers:   Property 'Items' does not exist on type 'BatchExecuteStatementCommandOutput'.
myapi-typescript-handlers: src/get-shopping-lists.ts(41,32): error TS7006: Parameter 'item' implicitly has an 'any' type.
myapi-typescript-handlers: src/get-shopping-lists.ts(46,29): error TS2339: Property 'LastEvaluatedKey' does not exist on type 'ServiceOutputTypes'.
myapi-typescript-handlers:   Property 'LastEvaluatedKey' does not exist on type 'BatchExecuteStatementCommandOutput'.
myapi-typescript-handlers: src/get-shopping-lists.ts(46,65): error TS2339: Property 'LastEvaluatedKey' does not exist on type 'ServiceOutputTypes'.
myapi-typescript-handlers:   Property 'LastEvaluatedKey' does not exist on type 'BatchExecuteStatementCommandOutput'.

I ended up tweaking code in get-shopping-lists.ts per below, just to get pdk build to work. Not sure if this will break things later on or not.

const shoppingListId = request.input.requestParameters.shoppingListId;
  // const Command = shoppingListId ? QueryCommand : ScanCommand;

  const response = await dbclient.send(new ScanCommand({

Also tutorial didn't discuss how to handle various test-files generated under root-directory/packages/api/handlers/typescript/test. They contain notes like // TODO: Update the test as appropriate when you implement your handler, but some guidance on approach to do this would be good. Ended up changing extension of tests to *._ts so that they would be skipped.

Things were mostly working for me in the Shopping List tutorial up to Submodule 4, but then the code changes suggested in part 4 somehow broke things. I can't really understand why.

The pdk build from the root directory is working, I am able to pdk deploy:dev from packages/infra ... and I have updated per here, the credentials locally with:

curl https://aws cloudformation describe-stacks --query "Stacks[?StackName=='infra-dev'][].Outputs[?contains(OutputKey, 'WebsiteDistributionDomainName')].OutputValue" --output text/runtime-config.json > packages/website/public/runtime-config.json
I also found in the code that it somehow expected that runtime-config.json at packages/website/src/components/RuntimeContext/ too, so I ran:

curl https://aws cloudformation describe-stacks --query "Stacks[?StackName=='infra-dev'][].Outputs[?contains(OutputKey, 'WebsiteDistributionDomainName')].OutputValue" --output text/runtime-config.json > packages/website/src/components/RuntimeContext/runtime-config.json

But still no joy.

• Can someone confirm that this tutorial still works end-to-end?
• There were some switches in naming around MyApi and ApiConstruct between parts 1 and 2, but I stuck with the MyApi naming and just adjusted some code snippets from part 2 accordingly.
• Rather than start from scratch, are there ways to regenerate boiler plate for packages/api, without deleting everything?
• Also, what is the strategy to purge some disk used to pdk build each time. It seems the size of my node_modules in the root directory is growing all the time. Now at 2.8GB?

Meanwhile, there is another 1.1GB in the pnpm path at /home/userXXX/.local/share/pnpm/store/v3/. This is all related to this PDK repo, since I'm not using pnpm for anything else.

[agdimech]

Hi!

Thanks for raising this very detailed posted, greatly appreciated!

Firstly apologies that you ran into these issues - it seems the aws sdk was updated which resulted into the build issues you were facing. I have since raised #744 which updates the docs to remedy this. I have also added samples for the unit tests which show how you can mock the dynamo client and also ensure consistent naming i.e: api.ts -> apis/myapi.ts. I have also re-run the whole example from start to finish using the updated PR docs and can confirm this all works now :)

The curl command only needs to be run from the root of the monorepo and should end up with the following file: packages/website/public/runtime-config.json. I have run this and confirm this works.

In terms of your node_modules size increasing, that is a side effect of NX as for every build which results in package changes it will store an entry in node_modules/.cache/nx. This is used as an optimization so that you can skip having to rebuild something if it hasn't changed. The easiest way to prune this is to either rm -rf node_modules/.cache/nx or you can run pdk nx reset.

[Analect]

@agdimech ... I have been through your updates. The main change I saw was adding a root-folder/packages/api/handlers/typescript/src/dynamo-client.ts file (per below) and then referencing this from each of the handler files in ``root-folder/packages/api/handlers/typescript/src/` ... and your changes around handling DynamoDB scan/queries.

import { DynamoDBClient } from '@aws-sdk/client-dynamodb';

export const ddbClient = new DynamoDBClient({ region: process.env.AWS_REGION });

On deployment, I am able to access the apiExplorer endpoint.

But not the default home Shopping Lists .. where I continue to get this error around DefaultApiClient missing, which I can't comprehend.

I know the standard procedure with CDK is to grant full admin access, but I choose to manage things in a more granular way and try to grant permissions widely only at a service level, per the extract below for a CDK execution policy. This works, as the stack deploys without error. However, could I be missing some permissioning or role that controls how this TypeSafeApi works?

"Statement": [
      {
        "Effect": "Allow",
        "Action": [
          "apigateway:*",
          "cloudwatch:*",
          "lambda:*",
          "logs:*",
          "s3:*",
          "ssm:*",
          "bedrock:*",
          "cognito-idp:*",
          "codebuild:*",
          "sns:*",
          "cognito-identity:*",
          "dynamodb:*"
        ],
        "Resource": "*",
        "Condition": {
          "StringEquals": {
            "aws:RequestedRegion": "eu-central-1"
          }
        }
      },
{
        "Effect": "Allow",
        "Action": [
          "cloudfront:*",
          "wafv2:*"
        ],
        "Resource": "*"
      },

Maybe as a way of determining what might be missing, does your full stack deploy 60 resources like mine?

[agdimech]

You were quite unlucky to encounter this issue as there was a bug in the sample code that we vend from the website that we recently fixed to resolve this. Unfortunately resynthing won't fix it as sample code is only generated once.

To resolve this, modify your index.tsx as follows:

Add an import:
import TypeSafeApiClientProvider from "./components/TypeSafeApiClientProvider";

Wrap your app in the provider:

<TypeSafeApiClientProvider>
       <App />
</TypeSafeApiClientProvider>

[Analect]

@agdimech ... just for the avoidance of doubt, root-repo/packages/website/src/index.tsx should look like this?

SPDX-License-Identifier: Apache-2.0 */
import { NorthStarThemeProvider } from "@aws-northstar/ui";
import React from "react";
import { createRoot } from "react-dom/client";
import { BrowserRouter } from "react-router-dom";
import Auth from "./components/Auth";
import RuntimeContextProvider from "./components/RuntimeContext";
import App from "./layouts/App";
import TypeSafeApiClientProvider from "./components/TypeSafeApiClientProvider";

createRoot(document.getElementById("root")!).render(
  <React.StrictMode>
    <NorthStarThemeProvider>
      <BrowserRouter>
        <RuntimeContextProvider>
          <Auth>
            <TypeSafeApiClientProvider>
              <App />
            </TypeSafeApiClientProvider>
          </Auth>
        </RuntimeContextProvider>
      </BrowserRouter>
    </NorthStarThemeProvider>
  </React.StrictMode>,
);

[Analect]

I can confirm this is working now. I would never have figure that out myself. Thanks!

[agdimech]

Fantastic - happy coding :)

[agdimech]

@agdimech ... thanks so much for your responsiveness with this. I'm glad I'm not certifiably mad, just yet, at least. I'll give this a try again to get things working. Thanks also for the info on nx. I haven't worked with it before.

Just as an aside, I was looking for a few other implementations to look at, to get a better handle on PDK. There aren't that many out there I've been able to find ... other than that of your colleague @cogwirrel at aws-textract-document-data-extraction-platform. That seems to use a aws-amplify/ui-react front-end, which is perhaps less daunting than the cloudscape tooling in this project ... which is a whole new front-end to master! Are there others you can point to?

I'm interested in implementing something like your US-based colleagues have done here, leveraging generative-ai-cdk-constructs, but with some of the guard-rails offered by PDK. I'm not sure if it's straight-forward to tap into these CDK constructs directly with PDK or if there is a requirement for some extra level of abstraction for them to be somehow wrapped for consumption with PDK ... for instance in part 1 of the tutorial when @aws/pdk/type-safe-api was added to the project, that brought with it a huge amount of boiler-plate. Is something similar required when working with the genai-cdk-constructs, for instance? I've experimented with having sub-projects within projects using projen directly, but I quickly got tangled up ... something that PDK was somehow better able to manage.

One final question - I had been using another tool recently from one of your colleagues in Japan to build a backend and frontend together, leveraging codebuild and pushing to cloudfront. If I was to get that working in conjunction with a PDK project, that's going to require a build image with alot more tooling on-board (including Java). I noticed there was a docker capability recently introduced in this repo here. Would I need to leverage that somehow for deploy-time-build to work?lang=typescript).

Keep up the great work. Colum

Not a problem at all!

In terms of pointing to other references, there is this which you can reference: https://github.com/awslabs/threat-composer

From a PDK/Projen perspective, it just synthesizes any kind of file so pretty much the sky is the limit with what you can do. If you simply want to vend some sample CDK code as an app/lib you can easily do that with Projen/PDK. Calling CDK constructs directly from Projen/PDK on the other hand doesn't make a lot of sense as they have very different concerns.

I haven't used deploy-time-build but I would just warn that whenever you plan on building something at deployment time, you run the risk of not being able to have repeatable builds/deployments if reverting or retrying. A good example is global libs changing between different deployment worker images or even worse not using pinned dependencies. This could result in different code being deployed for the same source reference (i.e: prod code is different to your staging env). In my humble opinion, it is much safer to have a discrete build step that generates artifacts (i.e: cloudformation, cloud assembly, etc), test it, then deploy it. To answer your actual question, I have created a docker image which bundles all the required PDK dependencies which you are welcome to try: public.ecr.aws/p9i6h6j0/aws-pdk:latest

[Analect]

Calling CDK constructs directly from Projen/PDK on the other hand doesn't make a lot of sense as they have very different concerns.

Perhaps I misunderstood, but I thought a big thrust behind these tools was to aid the building-out of boilerplate around infrastructure capabilities wrapped as CDK constructs.

[agdimech]

Ahh yes that is correct - sorry I must have misread your question, that is the exact intent.

TypeSafeAPI is a good example in that it does the undifferentiated heavy lifting for you by vending both CDK and Lambda wrappers (whilst hooking them up for you). You could adopt a similar approach to building your own constructs for example a step function construct that takes a workflow definition and generates all the handlers, clients, etc. Or in your case a GenAI construct to vend whatever boilerplate is required :)