Test Framework Status

[Mr-Nobody21]

Test Framework Development
Done :
Cosmos Implementation(Implemented chains.Chain interface)
xCall requirement collection (fee part excluded)
Risk and Assumptions
Record test cases (IBC integration excluded)
Make sure the framework works

To Be Done :
Xcall fee design and requirement
xCall IBC integration
Code refactor to keep common steps in executor and xcall specific steps to xcall_test
To execute multiple scenarios at same time(Concurrency)
Icon chain implementation and make sure test scripts run for Icon chain too without any changes
Testnet implementation for Cosmos and Icon

[AntonAndell]

Few things i think would be needed aswell:
Add a new method to the chain interface to setup IBC (setupIBC) that deploys relevant contract that is going to be mandatory for everything to function. For ICON this would be xCall, ibc handler, a client and a base dapp. Would have to be specified in config.

Develop a sample xCall dapp. that we can add base functions. Send message, rollback etc. Something like https://github.com/icon-project/btp2-java/blob/c6553543f864477d7f229af59630703c760ebb5a/xcall/src/test/java/foundation/icon/btp/xcall/sample/DAppProxySample.java

Implement relevant xCall and sample dapp methods into the chain interface.

[AntonAndell]

Currently there are also some adding of chains also which needs to be moved into the executor for integration test. Then all the chain setup should be re usable for e2e just configure a different client.

[AntonAndell]

@hemz10 How are we looking for deploying contracts time wise. If XCall is integrated with IBC then 4 contracts are deployed in total to test xCall. But if we redeploy for each test that is quite a lot of deployments. Do you think it will be a big performance loss? and what are our options to improve this?

[hemz10]

@AntonAndell , Sure we can add relevant methods to interface.
Adding of chains is currently in executor for integration test but we can't use that for e2e as for e2e we need couple of chains and an relayer running, so for e2e we will have a separate executor and chain setup.
As for now to test admin we need to deploy contracts for each scenario so that the state is new, when there are multiple contracts that needs to be deployed then we can use the same contract for testing all scenario which is possible. If there is performance issue we can improve by deploying single contract for all scenario.

[CyrusVorwald]

This doesn't need a framework.

If you want to have extensive testing, extend https://github.com/strangelove-ventures/interchaintest

If you want to get something done quickly, just write scripts that spin up the components and do a test. For reference, see https://github.com/datachainlab/yui-relayer-build/tree/v0.2/tests

I would just directly copy from https://github.com/icon-project/btp2/tree/main/e2edemo to work in this situation

[AntonAndell]

I agree we could probably make stuff a lot easier for ourselves. Move away for extensive testing of xCall and focus it on IBC.

Extension of https://github.com/strangelove-ventures/interchaintest is what has been done but with modification i think? Why can't we use it as is? @MuhammedIrfan

[MuhammedIrfan]

I agree we could probably make stuff a lot easier for ourselves. Move away for extensive testing of xCall and focus it on IBC.

Extension of https://github.com/strangelove-ventures/interchaintest is what has been done but with modification i think? Why can't we use it as is? @MuhammedIrfan

It was due to https://github.com/strangelove-ventures/interchaintest/blob/main/chainfactory.go#L127 which expects the chain to implement ibc.Chain interface. Hence we have implemented https://github.com/icon-project/IBC-Integration/blob/main/test/chains/icon/icon_chain.go#L24

[CyrusVorwald]

Does this mean that custom tests can now be written for the relevant chains as defined by https://github.com/strangelove-ventures/interchaintest/blob/main/docs/writeCustomTests.md ?

[MuhammedIrfan]

Yes @CyrusVorwald

[CyrusVorwald]

Thanks for clarifying

[redlarva]

@MuhammedIrfan
It would be helpful if you could provide us with:

1. xCall IBC integration : Could you please provide us with a test flow that illustrates communication between the relayer and Archway chain, even if it is just a test that covers the happy path? We understand that the relayer is not currently functional, but since the relay interfaces have been defined, having a test case that utilizes these interfaces would still be beneficial in helping us to understand how you intended to test the integration between the relayer and the chain.

2. Icon chain implementation and make sure test scripts run for Icon chain too without any changes

[MuhammedIrfan]

@MuhammedIrfan It would be helpful if you could provide us with:

1. xCall IBC integration : Could you please provide us with a test flow that illustrates communication between the relayer and Archway chain, even if it is just a test that covers the happy path? We understand that the relayer is not currently functional, but since the relay interfaces have been defined, having a test case that utilizes these interfaces would still be beneficial in helping us to understand how you intended to test the integration between the relayer and the chain.

At present the relay has not been integrated, it need to be integrated to e2e test, The integration test is capable of testing only smart contracts deployed to a single chain.

2. Icon chain implementation and make sure test scripts run for Icon chain too without any changes

Sure @redlarva, The icon chain is being implemented here and it is expected to run test scripts without any changes.

[pragyanshrestha-ibriz]

@MuhammedIrfan is it possible to clarify on how the current framework was being planned to use with the E2E test? Mainly how the testcase were being planned to run to check if the Relay and Archway component's integration. The test does not need to be functional but just an example or some testcase that covers a few components would be really helpful.

[hemz10]

Hi @pragyanshrestha-ibriz , these are some e2e scenarios we tested from cosmos to cosmos based chain. These will cover creating clients, paths between chains and relaying packets across chains. We can have even more detailed steps in the scenarios. But just to make sure we can cover e2e we did this poc.

Scenario: Sending token transfer over IBC
Given couple of IBC chains running
And user wallet is funded 
When we send IBC transfer and relay packets
Then funds should be transferred and amount should be debitted from account

Scenario: sending amount more than balance
Given couple of IBC chains running
And user wallet is funded
When we send amount greater than balance
Then transaction should fail

Scenario: Sending negative amount for transer
Given couple of IBC chains running
And user wallet is funded
When we send negative amount for transer
Then transaction should fail