Apply Back-pressure to outgoing /bulkTransaction requests from SDK-Scheme-Adapter to Switch

[mdebarros]

Goal:

As a Payer FSP

I want to ensure there is adequate Back-pressure when sending associated requests from a /bulkTransaction processing request

so that the Mojaloop Switch is able to process the individual Discovery requests, Bulk Requests for Quotes and Transfers in a consistent predictable manner.

Acceptance Criteria:

• Characterization of the SDK-Scheme-Adapter utilising a Platformed-shared-bc Kafka Consumer library that supports Back-pressure when consuming messages from the Kafka Topic

Complexity: High

Uncertainty: Medium

---

Tasks:

• Update Platformed-shared-bc Kafka Consumer library
• Test Platformed-shared-bc Kafka Consumer library
• Created a snapshot release for the Platformed-shared-bc Kafka Consumer library
• Update SDK-Scheme-Adapter to utilise the modified Platformed-shared-bc Kafka Consumer library
• Run Characterization locally to compare Pre and Post changes to the Platformed-shared-bc Kafka Consumer library
• Determine if Characterization should be done remotely on a similiar platform to the previous Characterization effort.

Done

• Acceptance Criteria pass
• Designs are up-to date
• Unit Tests pass
• Integration Tests pass
• Code Style & Coverage meets standards
• Changes made to config (default.json) are broadcast to team and follow-up tasks added to update helm charts and other deployment config.
• ...

Pull Requests:

• feat(mojaloop/#3026): apply back-pressure to outgoing /bulkTransaction requests from SDK-Scheme-Adapter to Switch platform-shared-lib#6
• feat(mojaloop/#3026): apply back-pressure to outgoing /bulkTransaction requests from SDK-Scheme-Adapter to Switch sdk-scheme-adapter#426

Follow-up:

• N/A

Dependencies:

• N/A

Accountability:

• Owner: @mdebarros
• QA/Review: TBC

[mdebarros]

Observations

• Test with existing SDK-Scheme-Adapter v21.4.0 with Kafka Consumer consuming as many messages as possible with no lock-stop processing resulted in a processing time of 37768ms with some failures. (ref: 20221201-std-80-err-TTK-Assertion-Report-multi-2022-12-01T09 29 33.478Z.html)
• Test with modified SDK-Scheme-Adapter v21.6.0-snapshot.0 with Kafka Consumer providing Back-pressure by consuming a single message at a time resulted in a bulkTransaction with 80 individual transfers taking 22585ms with no failures. (ref: 20221201-1msg-80-TTK-Assertion-Report-multi-2022-12-01T12 35 35.016Z.html)

Test-Reports.zip

Findings

• The above preliminary results indicate that Back-pressure would improve the overal performance characteristcs of a bulkTransaction request through a live v14.1.0 Mojaloop deployment.

Recommendations

• Finalise the Kafka Consumer changes with an official release and update the SDK-Scheme-Adapter to support those changes.
• Re-run the characterization using a purpose build environment for Performance testing
• Re-run the characterization using the previous methodology used for demo'ing the SDK's capabilities through a simulated TTK switch

Code Branches

• https://github.com/mojaloop/sdk-scheme-adapter/tree/feat/%233026-apply-back-pressu[…]oing-bulkTransaction-requests-from-SDK-Scheme-Adapter-to-Switch
• https://github.com/mdebarros/platform-shared-lib/tree/feat/%233026-apply-back-pres[…]oing-bulkTransaction-requests-from-SDK-Scheme-Adapter-to-Switch

[mdebarros]

Refer to the following comment for next steps: #3026 (comment)