-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Shopify acceptance tests fail and block the PRs #197
Comments
The problem is caused by the fact that I used:
# just for the current session, until restart
sudo modprobe iptable_nat
# include the nat module to survive restart
echo iptable_nat | sudo tee -a /etc/modules-load.d/modules This solves the problem. |
The tests still fail. These are:
All these tests seem to be flaky, dependent on the infrastructure and the development practices. They seem to be not related to the CDK changes, but blocking them. |
rpopov
changed the title
Shopify acceptance tests fail due to Dagger/Buildkit container failure
Shopify acceptance tests fail and block the PRs
Jan 8, 2025
rpopov
added a commit
to rpopov/airbyte-python-cdk
that referenced
this issue
Jan 13, 2025
At record extraction step, in each record add the service field $root holding a reference to: * the root response object, when parsing JSON format * the original record, when parsing JSONL format that each record to process is extracted from. More service fields could be added in future. The service fields are available in the record's filtering and transform steps. Avoid: * reusing the maps/dictionaries produced, thus avoid building cyclic structures * transforming the service fields in the Flatten transformation. Explicitly cleanup the service field(s) after the transform step, thus making them: * local for the filter and transform steps * not visible to the next mapping and store steps (as they should be) * not visible in the tests beyond the test_record_selector (as they should be) This allows the record transformation logic to define its "local variables" to reuse some interim calculations. The contract of body parsing seems irregular in representing the cases of bad JSON, no JSON and empty JSON. Cannot be unified as that that irregularity is already used. Update the development environment setup documentation * to organize and present the setup steps explicitly * to avoid misunderstandings and wasted efforts. Update CONTRIBUTING.md to * collect and organize the knowledge on running the test locally. * state the actual testing steps. * clarify and make explicit the procedures and steps. The unit, integration, and acceptance tests in this exactly version succeed under Fedora 41, while one of them fails under Oracle Linux 8.7. not related to the contents of this PR. The integration tests of the CDK fail due to missing `secrets/config.json` file for the Shopify source. See airbytehq#197
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Running the CAT tests locally on:
Observations
fails with the container (Docker image: registry.dagger.io/engine v0.15.1) errors:
Namely:
/usr/sbin/iptables -t nat -S --wait
command fails with exit code 3 which does not allow the container to initialize and terminates it.man iptables
states: "Errors which indicate an incompatibility between kernel and user space cause an exit code of 3".ulimit -n 1048576
used to fail when used podman instead of docker.The same failure seems to happen in the pipelines of the Pull requests. Examples:
extra_fields
inglobal_substream_cursor
#195The logs report the same:
CustomSchemaNormalization
#194Async Retriever
change url path for download retriever #192Suggestion
with instructions how to resolve those issues locally.
The text was updated successfully, but these errors were encountered: