Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(model-transformer) IndexName -> index in query list resolver #2912

Merged
merged 2 commits into from
Oct 2, 2024

Conversation

atierian
Copy link
Member

@atierian atierian commented Sep 27, 2024

Description of changes

Renames invalid IndexName key to valid index for generated model list queries.

The VTL data resolver for DynamoDB data sources in generated model list queries contains a code path that (as far as I can tell) is never hit.

#if( !$util.isNull($ctx.stash.metadata.index) )
  #set( $ListRequest.IndexName = $ctx.stash.metadata.index )
#end

I'm confident it's never hit because IndexName is an invalid key to include in the request function evaluation for the only two DDB operations this resolver function can initiate -- Scan and Query. The correct key is index.

Mapping Document Structure

AppSync Documentation References

The linked AppSync documentation shows that the correct key name for this purpose is index, not IndexName.

Query Mapping Document Structure
{
    "version" : "2017-02-28",
    "operation" : "Query",
    "query" : {
        "expression" : "some expression",
        "expressionNames" : {
            "#foo" : "foo"
        },
        "expressionValues" : {
            ":bar" : ... typed value
        }
    },
    "index" : "fooIndex", // <--- 👀 
    "nextToken" : "a pagination token",
    "limit" : 10,
    "scanIndexForward" : true,
    "consistentRead" : false,
    "select" : "ALL_ATTRIBUTES" | "ALL_PROJECTED_ATTRIBUTES" | "SPECIFIC_ATTRIBUTES",
    "filter" : {
        ...
    },
    "projection" : {
        ...
    }
}
Scan Mapping Document Structure
{
    "version" : "2017-02-28",
    "operation" : "Scan",
    "index" : "fooIndex", // <--- 👀 
    "limit" : 10,
    "consistentRead" : false,
    "nextToken" : "aPaginationToken",
    "totalSegments" : 10,
    "segment" : 1,
    "filter" : {
        ...
    },
    "projection" : {
        ...
    }
}

For completeness sake, here's the error message if ctx.stash.metadata.index exists and $ListRequest.IndexName is set.

{
  "data": {
    "listThings": null
  },
  "errors": [
    {
      "path": [
        "listThings"
      ],
      "data": null,
      "errorType": "MappingTemplate",
      "errorInfo": null,
      "locations": [
        {
          "line": 2,
          "column": 3,
          "sourceName": null
        }
      ],
      "message": "Unsupported element '$[IndexName]'."
    }
  ]
}

Q&A

If this code path is never hit, why are you making this change?

I'd like to use it 😄
For the conversation transformer, we use the model generated list query to surface conversation messages. It's important that message history be returned in sorted order. By fixing this issue, I can insert a simple resolver function into the existing list query pipeline to set ctx.stash.metadata.index.

Ok, but why is it IndexName now if that doesn't work?

As far as I can tell, it's been IndexName since this feat: transformer redesign PR in the aws-amplify/amplify-cli repo. I wasn't able to find any relevant commit or PR descriptions, or discussions from reviewers around this line. My assumptions are that IndexName was always incorrect, was added back then as a we might want this one day thing, and we've just never supported a code path that hits it.

It looks like this is the specific commit that added it back then.

I was unable to find anywhere in the transformer where ctx.stash.metadata.index is actually being set. Doing this exhaustively is hard though, so I can't make any assurances that it's not being done.

Model List Query Diff

## [Start] List Request. **
#set( $args = $util.defaultIfNull($ctx.stash.transformedArgs, $ctx.args) )
#set( $limit = $util.defaultIfNull($args.limit, 100) )
#set( $ListRequest = {
  "version": "2018-05-29",
  "limit": $limit
} )
#if( $args.nextToken )
  #set( $ListRequest.nextToken = $args.nextToken )
#end
#if( !$util.isNullOrEmpty($ctx.stash.authFilter) )
  #set( $filter = $ctx.stash.authFilter )
  #if( !$util.isNullOrEmpty($args.filter) )
    #set( $filter = {
  "and":   [$filter, $args.filter]
} )
  #end
#else
  #if( !$util.isNullOrEmpty($args.filter) )
    #set( $filter = $args.filter )
  #end
#end
#if( !$util.isNullOrEmpty($filter) )
  #set( $filterExpression = $util.parseJson($util.transform.toDynamoDBFilterExpression($filter)) )
  #if( $util.isNullOrEmpty($filterExpression) )
    $util.error("Unable to process the filter expression", "Unrecognized Filter")
  #end
  #if( !$util.isNullOrBlank($filterExpression.expression) )
    #if( $filterExpression.expressionValues.size() == 0 )
      $util.qr($filterExpression.remove("expressionValues"))
    #end
    #set( $ListRequest.filter = $filterExpression )
  #end
#end
#if( !$util.isNull($ctx.stash.modelQueryExpression) && !$util.isNullOrEmpty($ctx.stash.modelQueryExpression.expression) )
  $util.qr($ListRequest.put("operation", "Query"))
  $util.qr($ListRequest.put("query", $ctx.stash.modelQueryExpression))
  #if( !$util.isNull($args.sortDirection) && $args.sortDirection == "DESC" )
    #set( $ListRequest.scanIndexForward = false )
  #else
    #set( $ListRequest.scanIndexForward = true )
  #end
#else
  $util.qr($ListRequest.put("operation", "Scan"))
#end
#if( !$util.isNull($ctx.stash.metadata.index) )
-  #set( $ListRequest.IndexName = $ctx.stash.metadata.index )
+  #set( $ListRequest.index = $ctx.stash.metadata.index )
#end
$util.toJson($ListRequest)
## [End] List Request. **

CDK / CloudFormation Parameters Changed

Issue #, if available

Description of how you validated changes

Checklist

  • PR description included
  • yarn test passes
  • E2E test run linked
  • Tests are changed or added
  • Relevant documentation is changed or added (and PR referenced)
  • New AWS SDK calls or CloudFormation actions have been added to relevant test and service IAM policies
  • Any CDK or CloudFormation parameter changes are called out explicitly

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@atierian atierian changed the title fix(model-transformer) IndexName -> index in otherwise unused query list resolver fix(model-transformer) IndexName -> index in query list resolver Sep 27, 2024
@atierian atierian marked this pull request as ready for review September 27, 2024 11:59
@atierian atierian requested a review from a team as a code owner September 27, 2024 11:59
@atierian atierian merged commit 942df72 into main Oct 2, 2024
11 of 12 checks passed
@atierian atierian deleted the fix-query-list-index-name branch October 2, 2024 17:43
palpatim added a commit that referenced this pull request Oct 10, 2024
* chore: update .jsii assembly

* chore: update .jsii assembly

* chore: migrate pg array objects e2e test in gen2 cdk (#2906)

* chore: graphql prep for test migration

* refactor: generic graphql field selection string with fieldmap

* feat: add postgres array objects e2e test

* test: remove bootstrap in test code

* chore: schema cleanup

* chore: final cleanup

* chore: add explanation on FieldMap ans examples

* chore: remove dup test

---------

Signed-off-by: Kevin Shan <[email protected]>
Co-authored-by: Tim Schmelter <[email protected]>

* fix(model-transformer) IndexName -> index in query list resolver (#2912)

* chore: upgrade cdk library dependency to 2.158.0 (#2876)

* chore: upgrade cdk dependency to 2.158.0

* chore: install and use nvm

* chore: use full version for nvm

* chore: testing linux build with nvm

* chore: fix version in cdk tests

* chore: update jsii files

* update: increase memory size

* add: debug statement

* update: mem size back to 8096, use ps1 file for shell script

* fix: path to Setup-NodeVersion.ps1

* fix: path to codebuild_specs/Setup-NodeVersion.ps1

* add: set runtime version

* update: image

* add: debug statement

* update: use earlier code

* add: debug statements

* update: clean up code

* update: use the correct image

* add: list installed node versions and used nodejs.install

* restart: install nvm using choco

* add: back mem size variable

* add: nvm install and use 18.20.4

* add: env var NVM_HOME and NVM_SYMLINK

* add: spawn powershell as admin

* update: remove all other builds

* add: debug statement

* add: env var path

* update: print env var

* add: commands

* update: env var set up

* add: refresh env var

* update: more debug statement

* update

* revamp: find nvm.exe

* update: install nvm windows directly

* update: launch new shell if current shell does not recognize nvm

* update: install node in buildspec

* add: install and use node in build spec

* update: use single quote to prevent interpreting \

* add: 2 scripts, one for installing nvm, another for using nvm

* fix: path error

* test: which way set env var

* update: set up env var in pre_build

* update: use choco in pre-build

* fix: syntax error

* update: build_windows working, running all tests

* test: remove bootstrap in test code

* debug: _runGqlE2ETests

* update: debug_workflow

* update: debug_workflow

* update: debug_workflow

* update: debug_workflow

* add: debug statement

* add: debug test

* add: debug

* update: use uuid for bucket name

* remove: use of uuid

* add: debug statement

* update: use differrent bucket name

* add: mili second timestamp

* add: debug statement

* remove: debug statement

* remove: redundant code

---------

Co-authored-by: Bobby Yu <[email protected]>
Co-authored-by: Tim Schmelter <[email protected]>

* test: fix gen 1 init (#2924)

* fix(conversation): allow changes to systemPrompt, inferenceConfig, aiModel to be hotswapped (#2923)

* feat: auto increment support (#2883)

* chore(graphql-default-value-transformer): tidy tests

* test(graphql-default-value-transformer): add unit tests for auto increment support

* feat: 🎸 utils to detect Postgres datasource

* feat: 🎸 support auto increment

Implements support for auto increment (serial) fields from Postgres
datasources. Such fields are denoted by an empty `@default` applied to
an `Int` field.

* test(graphql-default-value-transformer): pk can be auto increment

* test(graphql-default-value-transformer): auto-increment crud e2e

* chore: describe test purpose

* chore: removing logging

* chore: describe why invalid cases are invalid

* chore: remove unecessary e2e test case

* chore: test messaging clarity

* chore: type safety

* chore: alphabetize list

* chore: type of return value asserts against string

Co-authored-by: Tim Schmelter <[email protected]>

* chore: test ensures customers can insert to serial fields with custom values

* chore: verify that @default(value) works on mysql

* chore: remove unecessary ssm test case

* chore: update branch from main

* test: value cannot be null on ddb

---------

Co-authored-by: Tim Schmelter <[email protected]>

* fix(conversation): use functionMap for custom handler IFunction reference (#2922)

* fix(generation): gracefully handle stringified tool_use responses (#2919)

* feat(conversation): per message items and lambda history retrieval pattern (#2914)

* fix: sql default value e2e failures (#2932)

* fix(generation): remove trailing comma in inferenceConfig resolver code (#2933)

* fix: add aws_iam to custom operations when enableIamAuthorization is enabled; fix graphql type utils (#2921)

- test: Add additional tests to fix coverage metrics for unchanged files
- test: Add implicit IAM auth support tests
  - Added a skipped test for custom type support, to be re-enabled once we
    figure out the right strategy for this.

* fix: appsync ttl correct duration time unit in ms (#2928)

Signed-off-by: Kevin Shan <[email protected]>

---------

Signed-off-by: Kevin Shan <[email protected]>
Co-authored-by: amplify-data-ci <[email protected]>
Co-authored-by: Kevin Shan <[email protected]>
Co-authored-by: Ian Saultz <[email protected]>
Co-authored-by: Phani Srikar Edupuganti <[email protected]>
Co-authored-by: Bobby Yu <[email protected]>
Co-authored-by: Dane Pilcher <[email protected]>
Co-authored-by: Peter V. <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants