[Helm] Helm test Cleanup is failing

[mdebarros]

Summary:

Helm test Cleanup is failing because the TTK Path.

❯ helm -n moja1 test moja1
NAME: moja1
LAST DEPLOYED: Fri May 19 14:09:33 2023
NAMESPACE: moja1
STATUS: deployed
REVISION: 1
TEST SUITE:     moja1-ml-ttk-test-cleanup
Last Started:   Fri May 19 14:37:42 2023
Last Completed: Fri May 19 14:37:50 2023
Phase:          Failed
TEST SUITE:     moja1-ml-ttk-test-setup-sdk-bulk
Last Started:   Fri May 19 14:34:09 2023
Last Completed: Fri May 19 14:34:54 2023
Phase:          Succeeded
TEST SUITE:     moja1-ml-ttk-test-setup-tp
Last Started:   Fri May 19 14:33:06 2023
Last Completed: Fri May 19 14:33:50 2023
Phase:          Succeeded
TEST SUITE:     moja1-ml-ttk-test-setup
Last Started:   Fri May 19 14:25:41 2023
Last Completed: Fri May 19 14:27:21 2023
Phase:          Succeeded
TEST SUITE:     moja1-ml-ttk-test-val-bulk
Last Started:   Fri May 19 14:30:50 2023
Last Completed: Fri May 19 14:33:06 2023
Phase:          Succeeded
TEST SUITE:     moja1-ml-ttk-test-val-gp
Last Started:   Fri May 19 14:27:21 2023
Last Completed: Fri May 19 14:30:50 2023
Phase:          Succeeded
TEST SUITE:     moja1-ml-ttk-test-val-sdk-bulk
Last Started:   Fri May 19 14:34:54 2023
Last Completed: Fri May 19 14:37:41 2023
Phase:          Succeeded
TEST SUITE:     moja1-ml-ttk-test-val-tp
Last Started:   Fri May 19 14:33:50 2023
Last Completed: Fri May 19 14:34:09 2023
Phase:          Succeeded
NOTES:
Thank you for installing mojaloop.

Your release is named: moja1.

Mojaloop Official Website: https://mojaloop.io

To learn more about Mojaloop: https://docs.mojaloop.io
To learn more about Mojaloop deployments: https://docs.mojaloop.io/legacy/deployment-guide
To learn more about this release: https://github.com/mojaloop/helm/releases/tag/v15.0.0

This Helm chart lives @ http://github.com/mojaloop/helm

To learn more about this deployment, try:

  $ helm status moja1

Use the following command to execute Test cases:

  $ helm -n moja1 test moja1

Use the following command to execute Test cases and print logs to console:

  $ helm -n moja1 test moja1 --logs

Use the following command to execute a specific Test case:

  $ helm -n moja1 test moja1 --filter "name=moja1-<TEST_NAME_1>,name=moja1-<TEST_NAME_2>"

Use the following command to ignore a specific Test case:

  $ helm -n moja1 test moja1 --filter "!name=moja1-<TEST_NAME_1>"

View Test logs with the following commands:
  $ kubectl -n moja1 logs pod/moja1-ml-ttk-test-setup
  $ kubectl -n moja1 logs pod/moja1-ml-ttk-test-val-gp
  $ kubectl -n moja1 logs pod/moja1-ml-ttk-test-val-bulk
  $ kubectl -n moja1 logs pod/moja1-ml-ttk-test-setup-sdk-bulk
  $ kubectl -n moja1 logs pod/moja1-ml-ttk-test-val-sdk-bulk
  $ kubectl -n moja1 logs pod/moja1-ml-ttk-test-setup-tp
  $ kubectl -n moja1 logs pod/moja1-ml-ttk-test-val-tp
  $ kubectl -n moja1 logs pod/moja1-ml-ttk-test-cleanup

Copyright © 2020 Mojaloop Foundation
Error: pod moja1-ml-ttk-test-cleanup failed

Viewing the logs kubectl -n moja1 logs pod/moja1-ml-ttk-test-cleanup returns the following error:

> @mojaloop/[email protected] cli
> node src/client.js "-c" "cli-default-config.json" "-e" "cli-testcase-environment.json" "-i" "tmp_test_cases/testing-toolkit-test-cases-15.0.0/collections/hub/cleanup" "-u" "http://moja1-ml-testing-toolkit-backend:5050" "--report-format" "html" "--report-target" "s3://mojaloop-oss-qa-results/ttk-tests/reports/report.html" "--report-auto-filename-enable" "true" "--slack-webhook-url" "XXXX" "--extra-summary-information=Test Suite:Post Cleanup,Environment:[email protected]" "--save-report" "true" "--report-name" "post_cleanup" "--save-report-base-url" "testing-toolkit.local"
Listening on http://moja1-ml-testing-toolkit-backend:5050 outboundProgress events...
(node:22) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.
Please migrate your code to use AWS SDK for JavaScript (v3).
For more information, check the migration guide at https://a.co/7PzMCcy
(Use `node --trace-warnings ...` to show where the warning was created)
[Error: ENOENT: no such file or directory, stat 'tmp_test_cases/testing-toolkit-test-cases-15.0.0/collections/hub/cleanup'] {
errno: -2,
code: 'ENOENT',
syscall: 'stat',
path: 'tmp_test_cases/testing-toolkit-test-cases-15.0.0/collections/hub/cleanup'
}
TypeError: Cannot set properties of null (setting 'inputValues')
at Object.sendTemplate (/opt/app/src/modes/outbound.js:152:26)

Severity:
Medium

Priority:
Low

Expected Behavior

1. Helm test cleanup job should execute successfully when enabled

Steps to Reproduce

1. Install Mojaloop with Cleanup Test job enabled
2. Run helm test

Specifications

• Component (if known): Helm
• Version: v15.0.0
• Platform: moja1
• Subsystem: n/a
• Type of testing: manual
• Bug found/raised by: @mdebarros

Related to

• helm test errors in Mojaloop helm v15.0.0.0 (can't test 3ppi or bulk with helm test) helm#567
• Cleanup collection with SDK bulk tests is failing #3240

Notes:

• Severity when opened: Medium
• Priority when opened: Low

[mdebarros]

Created a follow-up story to address the AWS SDK warning in the log file --> #3352

[mdebarros]

Hey team! Please add your planning poker estimate with Zenhub @PaulGregoryBaker @vijayg10

[vijayg10]

The cleanup collection is missing in the testing-toolkit-testcases v15.0.0. There is a commit in the master with this collection, so re-released the version v15.0.0 by adding that commit.
https://github.com/mojaloop/testing-toolkit-test-cases/releases/tag/v15.0.0

[vijayg10]

The helm test is passing now

Test Suite:Post Cleanup
Environment:Development
┌───────────────────────────────────────────────────┐
│                      SUMMARY                      │
├───────────────────┬───────────────────────────────┤
│ Total assertions  │ 49                            │
├───────────────────┼───────────────────────────────┤
│ Passed assertions │ 49                            │
├───────────────────┼───────────────────────────────┤
│ Failed assertions │ 0                             │
├───────────────────┼───────────────────────────────┤
│ Total requests    │ 15                            │
├───────────────────┼───────────────────────────────┤
│ Total test cases  │ 4                             │
├───────────────────┼───────────────────────────────┤
│ Passed percentage │ 100.00%                       │
├───────────────────┼───────────────────────────────┤
│ Started time      │ Tue, 23 May 2023 09:07:49 GMT │
├───────────────────┼───────────────────────────────┤
│ Completed time    │ 2023-05-23T09:07:59.317Z      │
├───────────────────┼───────────────────────────────┤
│ Runtime duration  │ 10255 ms                      │
└───────────────────┴───────────────────────────────┘

[mdebarros]

The helm test is passing now

Test Suite:Post Cleanup
Environment:Development
┌───────────────────────────────────────────────────┐
│                      SUMMARY                      │
├───────────────────┬───────────────────────────────┤
│ Total assertions  │ 49                            │
├───────────────────┼───────────────────────────────┤
│ Passed assertions │ 49                            │
├───────────────────┼───────────────────────────────┤
│ Failed assertions │ 0                             │
├───────────────────┼───────────────────────────────┤
│ Total requests    │ 15                            │
├───────────────────┼───────────────────────────────┤
│ Total test cases  │ 4                             │
├───────────────────┼───────────────────────────────┤
│ Passed percentage │ 100.00%                       │
├───────────────────┼───────────────────────────────┤
│ Started time      │ Tue, 23 May 2023 09:07:49 GMT │
├───────────────────┼───────────────────────────────┤
│ Completed time    │ 2023-05-23T09:07:59.317Z      │
├───────────────────┼───────────────────────────────┤
│ Runtime duration  │ 10255 ms                      │
└───────────────────┴───────────────────────────────┘

Also confirmed that Clean-up is now working on Moja1 environment...

MojaBot
APP 12:53 PM

Testing Toolkit Report
Test Result: PASSED
Test Suite: Post Cleanup
Environment: [email protected]
Total assertions: 49
Passed assertions: 49
Failed assertions: 0
Total requests: 15
Total test cases: 4
Passed percentage: 100.00%
Started time: Tue, 23 May 2023 10:53:45 GMT
Completed time: 2023-05-23T10:53:55.652Z
Runtime duration: 10469 ms

View Report

[mdebarros]

The cleanup collection is missing in the testing-toolkit-testcases v15.0.0. There is a commit in the master with this collection, so re-released the version v15.0.0 by adding that commit. https://github.com/mojaloop/testing-toolkit-test-cases/releases/tag/v15.0.0

Also, no changes are needed to Mojaloop v15.0.0 release.

Existing Mojaloop v15.0.0 deployments should now be able to call the Cleanup script using helm test (or specifically helm test --filter "name=<RELEASE>-ml-ttk-test-cleanup")