Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deserialiser incorrectly deserialises empty NewImage Map and throws AttributeError #30

Open
ghost opened this issue Aug 9, 2021 · 0 comments

Comments

@ghost
Copy link

ghost commented Aug 9, 2021

This was an issue recently discovered when trying to set up a DynamoDB integration with Stitch. The stream was incorrectly configured to use ONLY_KEYS, however the error message received by Stitch was

Extraction failed for ‘NoneType’ object has no attribute ‘items'

this itself is a python error message received when the following in log_based.py line 90 is executed:
record_message = deserializer.deserialize_item(record['dynamodb'].get('NewImage'))

which calls

def deserialize_item(self, item):
        return self.deserialize({'M': item})

This makes boto3 TypeDeserializer deserialise a Map value like the following {'M': None} in the following Types.py function

def _deserialize_m(self, value):
        return dict([(k, self.deserialize(v)) for k, v in value.items()])

Thus causing the above error message.

To avoid this there should be type checks within the deserialize_item function and an appropriate call to the correct deserialize ie. {'Null': True} for None - since boto3 appears to require: "Value must be a nonempty dictionary whose key is a valid dynamodb type"

This also explains why the following check is never evaluated in log_base.py line 91:

if record_message is None:
                LOGGER.fatal('Dynamo stream view type must be either "NEW_IMAGE" "NEW_AND_OLD_IMAGES"')
                raise RuntimeError('Dynamo stream view type must be either "NEW_IMAGE" "NEW_AND_OLD_IMAGES"')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

0 participants