Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Working Relayer for Other chain -> MultiversX calls & MultiversX -> other chain #10

Merged
merged 37 commits into from
Nov 27, 2024
Merged
Changes from 8 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
aed8861
Update proto file definition.
raress96 Jun 10, 2024
76016bf
Update abi and classes to gateway v2 integration. Start working on up…
raress96 Jun 10, 2024
60f1290
Fix all tests and full update for Gateway V2.
raress96 Jun 11, 2024
57d941b
Update dependencies.
raress96 Jun 11, 2024
c85228c
Update error message.
raress96 Jun 19, 2024
67c7399
Refactor grpc verify to keep stream open.
raress96 Jun 27, 2024
a4aed05
Refactoring and fix tests for new grpc verify.
raress96 Jun 28, 2024
15ba736
Merge pull request #9 from multiversx/gateway_v2
raress96 Jul 3, 2024
e5b9cf3
Working axelar event processor and other chain -> MultiversX flow.
raress96 Jul 12, 2024
243a843
Update tests.
raress96 Jul 12, 2024
ddd39f4
Fix call to getPayload.
raress96 Jul 18, 2024
0172dfc
Working mvx to other chain event processing with log index support.
raress96 Aug 20, 2024
267be8f
Add tests for new event index support.
raress96 Aug 21, 2024
08781c3
Fix tests.
raress96 Aug 23, 2024
603f71b
Merge pull request #11 from multiversx/event_index_support
raress96 Aug 23, 2024
a940cfa
Remove grpc axelar api and change to rest gmp api. Get project to com…
raress96 Aug 30, 2024
ffd1414
Process tasks in approvals processor service.
raress96 Aug 30, 2024
20162c3
Handle events and send them to the gmp api.
raress96 Aug 30, 2024
39ac6ca
Refactor gateway and gas service processor. Handle all gas service ev…
raress96 Sep 2, 2024
2c6cd37
Remove unneded database entries.
raress96 Sep 2, 2024
e51fe45
Start working on tests.
raress96 Sep 2, 2024
b5d36fd
Fix existing tests and add new ones after refactoring to use axelar g…
raress96 Sep 4, 2024
af68f75
Working GMP API calls.
raress96 Sep 6, 2024
90e698e
Use base64 encoding for payload.
raress96 Sep 6, 2024
8bfa078
Update message executed event.
raress96 Sep 9, 2024
ce4dd19
Merge pull request #12 from multiversx/axelar_gmp_api_integration
raress96 Sep 11, 2024
05f234e
Fix relayer getting stuck if transaction returns error.
raress96 Sep 12, 2024
e5db7d4
Fix lint for e2e tests.
raress96 Sep 12, 2024
82fc7b1
Handle verify and refund tasks. Improvements for logging and small im…
raress96 Sep 26, 2024
cb4c9e0
Implement gas cost reporting to Amplifier API. Implement reporting of…
raress96 Sep 26, 2024
b4f84ec
Fix pipeline.
raress96 Sep 26, 2024
c505ba6
Merge pull request #13 from multiversx/gas_and_verify
raress96 Sep 30, 2024
20cb777
Take into account esdt issue cost as fee in case of its execute.
raress96 Oct 1, 2024
7ca3b38
Handle signers rotated event and updates.
raress96 Nov 15, 2024
0359052
handle all todos and check available gas fees before sending transact…
raress96 Nov 18, 2024
02920ba
Update gateway events after new abi and use v2 of cannot execute mess…
raress96 Nov 20, 2024
f17dc6d
Merge pull request #14 from multiversx/verify_worker_set_and_gas
raress96 Nov 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .env.example
Original file line number Diff line number Diff line change
@@ -7,9 +7,9 @@ REDIS_URL=127.0.0.1
EVENTS_NOTIFIER_URL=amqp://user:password@rabbitmq:5672
EVENTS_NOTIFIER_QUEUE=queue

CONTRACT_GATEWAY=erd1qqqqqqqqqqqqqpgqhxy6dv9k5p3u4d6rawnwjyp0j3sunu9dkklspga3t9
CONTRACT_GAS_SERVICE=erd1qqqqqqqqqqqqqpgqsrhknrwuvy606ar5l2kuaz4glgyyye9vkkls5ng86g
CONTRACT_ITS=erd1qqqqqqqqqqqqqpgqw08zahneragk9rnaujwe8qcyu84ehw2lkklsvca0jx
CONTRACT_GATEWAY=erd1qqqqqqqqqqqqqpgqvkhh6ex5m0sl0rgxn5790ljsscye0r48kkls7hrlaw
CONTRACT_GAS_SERVICE=erd1qqqqqqqqqqqqqpgq8fmglw6pngxsczmpa0kr3904shnclzsukklsjeykne
CONTRACT_ITS=erd1qqqqqqqqqqqqqpgqcv3rhjjrqpl88es4q25lw03hfhpw6s36kklsn6t9a6

CONTRACT_WEGLD_SWAP=erd1qqqqqqqqqqqqqpgqpv09kfzry5y4sj05udcngesat07umyj70n4sa2c0rp

1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -13,6 +13,7 @@ Based on Amplifier API Docs: https://bright-ambert-2bd.notion.site/Amplifier-API

1. Redis Server is required to be installed [docs](https://redis.io/).
2. PostgreSQL is required to be installed [docs](https://www.postgresql.org/).
3. For E2E tests you need dotenv-cli `npm install -g dotenv-cli`

In this repo there is a `docker-compose.yml` file providing these services so you can run them easily using `docker-compose up -d`

Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Locker } from '@multiversx/sdk-nestjs-common';
import { BinaryUtils, Locker } from '@multiversx/sdk-nestjs-common';
import { Inject, Injectable, Logger, OnModuleInit } from '@nestjs/common';
import { Cron } from '@nestjs/schedule';
import { GrpcService } from '@mvx-monorepo/common/grpc/grpc.service';
@@ -47,7 +47,7 @@ export class ApprovalsProcessorService implements OnModuleInit {

async handleNewApprovalsRaw() {
if (this.approvalsSubscription && !this.approvalsSubscription.closed) {
this.logger.log('GRPC approvals stream subscription is already running');
this.logger.debug('GRPC approvals stream subscription is already running');

return;
}
@@ -77,6 +77,8 @@ export class ApprovalsProcessorService implements OnModuleInit {
complete: onComplete.bind(this),
error: onError.bind(this),
});

this.logger.log('GRPC approvals stream subscription started successfully!');
}

async handlePendingTransactionsRaw() {
@@ -90,7 +92,7 @@ export class ApprovalsProcessorService implements OnModuleInit {
continue;
}

const { txHash, executeData, retry } = cachedValue;
const { txHash, externalData, retry } = cachedValue;

const success = await this.transactionsHelper.awaitSuccess(txHash);

@@ -106,7 +108,7 @@ export class ApprovalsProcessorService implements OnModuleInit {
}

try {
await this.executeTransaction(executeData, retry);
await this.executeTransaction(externalData, retry);
} catch (e) {
this.logger.error('Error while trying to retry Axelar Approvals response transaction...');
this.logger.error(e);
@@ -116,7 +118,7 @@ export class ApprovalsProcessorService implements OnModuleInit {
CacheInfo.PendingTransaction(txHash).key,
{
txHash,
executeData: executeData,
externalData: externalData,
retry: retry,
},
CacheInfo.PendingTransaction(txHash).ttl,
@@ -148,11 +150,15 @@ export class ApprovalsProcessorService implements OnModuleInit {
}
}

private async executeTransaction(executeData: Uint8Array, retry: number = 0) {
this.logger.debug(`Trying to execute Gateway execute transaction with executeData:`);
this.logger.debug(executeData);
private async executeTransaction(externalData: Uint8Array, retry: number = 0) {
// The Amplifier for MultiversX encodes the executeData as hex, we need to decode it to string
// It will have the format `function@arg1HEX@arg2HEX...`
const decodedExecuteData = BinaryUtils.hexToString(Buffer.from(externalData).toString('hex'));

this.logger.debug(`Trying to execute Gateway execute transaction with externalData:`);
this.logger.debug(decodedExecuteData);

const transaction = this.gatewayContract.buildExecuteTransaction(executeData, this.walletSigner.getAddress());
const transaction = this.gatewayContract.buildTransactionExternalFunction(decodedExecuteData, this.walletSigner.getAddress());

const gas = await this.transactionsHelper.getTransactionGas(transaction, retry);
transaction.setGasLimit(gas);
@@ -163,7 +169,7 @@ export class ApprovalsProcessorService implements OnModuleInit {
CacheInfo.PendingTransaction(txHash).key,
{
txHash,
executeData: executeData,
externalData,
retry: retry + 1,
},
CacheInfo.PendingTransaction(txHash).ttl,
Original file line number Diff line number Diff line change
@@ -11,6 +11,9 @@ import { Subject } from 'rxjs';
import { SubscribeToApprovalsResponse } from '@mvx-monorepo/common/grpc/entities/amplifier';
import { UserAddress } from '@multiversx/sdk-wallet/out/userAddress';
import { Transaction } from '@multiversx/sdk-core/out';
import { BinaryUtils } from '@multiversx/sdk-nestjs-common';

const mockExternalData = Buffer.from(BinaryUtils.stringToHex('approveMessages@61726731@61726732'), 'hex');

describe('ApprovalsProcessorService', () => {
let grpcService: DeepMocked<GrpcService>;
@@ -88,15 +91,15 @@ describe('ApprovalsProcessorService', () => {
walletSigner.getAddress.mockReturnValueOnce(userAddress);

const transaction: DeepMocked<Transaction> = createMock();
gatewayContract.buildExecuteTransaction.mockReturnValueOnce(transaction);
gatewayContract.buildTransactionExternalFunction.mockReturnValueOnce(transaction);

transactionsHelper.getTransactionGas.mockReturnValueOnce(Promise.resolve(100_000_000));
transactionsHelper.signAndSendTransaction.mockReturnValueOnce(Promise.resolve('txHash'));

// Process a message
const message: SubscribeToApprovalsResponse = {
chain: 'multiversx',
executeData: Uint8Array.of(1, 2, 3, 4),
executeData: mockExternalData,
blockHeight: 1,
};
observable.next(message);
@@ -109,8 +112,8 @@ describe('ApprovalsProcessorService', () => {
setTimeout(resolve, 500);
});

expect(gatewayContract.buildExecuteTransaction).toHaveBeenCalledTimes(1);
expect(gatewayContract.buildExecuteTransaction).toHaveBeenCalledWith(message.executeData, userAddress);
expect(gatewayContract.buildTransactionExternalFunction).toHaveBeenCalledTimes(1);
expect(gatewayContract.buildTransactionExternalFunction).toHaveBeenCalledWith('approveMessages@61726731@61726732', userAddress);
expect(transactionsHelper.getTransactionGas).toHaveBeenCalledTimes(1);
expect(transactionsHelper.getTransactionGas).toHaveBeenCalledWith(transaction, 0);
expect(transaction.setGasLimit).toHaveBeenCalledTimes(1);
@@ -123,7 +126,7 @@ describe('ApprovalsProcessorService', () => {
CacheInfo.PendingTransaction('txHash').key,
{
txHash: 'txHash',
executeData: message.executeData,
externalData: message.executeData,
retry: 1,
},
CacheInfo.PendingTransaction('txHash').ttl,
@@ -143,7 +146,7 @@ describe('ApprovalsProcessorService', () => {
const userAddress = UserAddress.fromBech32('erd1qqqqqqqqqqqqqpgqhe8t5jewej70zupmh44jurgn29psua5l2jps3ntjj3');
walletSigner.getAddress.mockReturnValueOnce(userAddress);
const transaction: DeepMocked<Transaction> = createMock();
gatewayContract.buildExecuteTransaction.mockReturnValueOnce(transaction);
gatewayContract.buildTransactionExternalFunction.mockReturnValueOnce(transaction);
transactionsHelper.getTransactionGas.mockRejectedValueOnce(new Error('Network error'));

await service.handleNewApprovalsRaw();
@@ -237,7 +240,7 @@ describe('ApprovalsProcessorService', () => {
redisCacheService.get.mockReturnValueOnce(
Promise.resolve({
txHash: 'txHashComplete',
executeData: Uint8Array.of(1, 2, 3, 4),
executeData: mockExternalData,
retry: 1,
}),
);
@@ -257,13 +260,13 @@ describe('ApprovalsProcessorService', () => {

it('Should handle retry', async () => {
const key = CacheInfo.PendingTransaction('txHashComplete').key;
const executeData = Uint8Array.of(1, 2, 3, 4);
const externalData = mockExternalData;

redisCacheService.scan.mockReturnValueOnce(Promise.resolve([key]));
redisCacheService.get.mockReturnValueOnce(
Promise.resolve({
txHash: 'txHashComplete',
executeData,
externalData,
retry: 1,
}),
);
@@ -273,7 +276,7 @@ describe('ApprovalsProcessorService', () => {
walletSigner.getAddress.mockReturnValueOnce(userAddress);

const transaction: DeepMocked<Transaction> = createMock();
gatewayContract.buildExecuteTransaction.mockReturnValueOnce(transaction);
gatewayContract.buildTransactionExternalFunction.mockReturnValueOnce(transaction);

transactionsHelper.getTransactionGas.mockReturnValueOnce(Promise.resolve(100_000_000));
transactionsHelper.signAndSendTransaction.mockReturnValueOnce(Promise.resolve('txHash'));
@@ -283,9 +286,9 @@ describe('ApprovalsProcessorService', () => {
expect(transactionsHelper.awaitSuccess).toHaveBeenCalledTimes(1);
expect(transactionsHelper.awaitSuccess).toHaveBeenCalledWith('txHashComplete');

expect(gatewayContract.buildExecuteTransaction).toHaveBeenCalledTimes(1);
expect(gatewayContract.buildExecuteTransaction).toHaveBeenCalledWith(
executeData,
expect(gatewayContract.buildTransactionExternalFunction).toHaveBeenCalledTimes(1);
expect(gatewayContract.buildTransactionExternalFunction).toHaveBeenCalledWith(
BinaryUtils.hexToString(externalData.toString('hex')),
userAddress,
);
expect(transactionsHelper.getTransactionGas).toHaveBeenCalledTimes(1);
@@ -300,7 +303,7 @@ describe('ApprovalsProcessorService', () => {
CacheInfo.PendingTransaction('txHash').key,
{
txHash: 'txHash',
executeData,
externalData,
retry: 2,
},
CacheInfo.PendingTransaction('txHash').ttl,
@@ -330,13 +333,13 @@ describe('ApprovalsProcessorService', () => {

it('Should handle retry error', async () => {
const key = CacheInfo.PendingTransaction('txHashComplete').key;
const executeData = Uint8Array.of(1, 2, 3, 4);
const externalData = mockExternalData;

redisCacheService.scan.mockReturnValueOnce(Promise.resolve([key]));
redisCacheService.get.mockReturnValueOnce(
Promise.resolve({
txHash: 'txHashComplete',
executeData,
externalData,
retry: 1,
}),
);
@@ -346,7 +349,7 @@ describe('ApprovalsProcessorService', () => {
walletSigner.getAddress.mockReturnValueOnce(userAddress);

const transaction: DeepMocked<Transaction> = createMock();
gatewayContract.buildExecuteTransaction.mockReturnValueOnce(transaction);
gatewayContract.buildTransactionExternalFunction.mockReturnValueOnce(transaction);

transactionsHelper.getTransactionGas.mockRejectedValueOnce(new Error('Network error'));

@@ -362,7 +365,7 @@ describe('ApprovalsProcessorService', () => {
CacheInfo.PendingTransaction('txHashComplete').key,
{
txHash: 'txHashComplete',
executeData,
externalData,
retry: 1,
},
CacheInfo.PendingTransaction('txHashComplete').ttl,
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
export interface PendingTransaction {
txHash: string;
executeData: Uint8Array;
externalData: Uint8Array;
retry: number;
}

This file was deleted.

Original file line number Diff line number Diff line change
@@ -4,7 +4,8 @@ import { Locker } from '@multiversx/sdk-nestjs-common';
import { ContractCallEventStatus } from '@prisma/client';
import { GrpcService } from '@mvx-monorepo/common';
import { ContractCallEventRepository } from '@mvx-monorepo/common/database/repository/contract-call-event.repository';
import { firstValueFrom } from 'rxjs';

const MAX_NUMBER_OF_RETRIES: number = 3;

@Injectable()
export class ContractCallEventProcessorService {
@@ -17,7 +18,7 @@ export class ContractCallEventProcessorService {
this.logger = new Logger(ContractCallEventProcessorService.name);
}

// Ofset at second 15 to not run at the same time as processPendingContractCallApproved
// Offset at second 15 to not run at the same time as processPendingMessageApproved
@Cron('15 */2 * * * *')
async processPendingContractCallEvent() {
await Locker.lock('processPendingContractCallEvent', async () => {
@@ -29,28 +30,23 @@ export class ContractCallEventProcessorService {
this.logger.log(`Found ${entries.length} ContractCallEvent transactions to execute`);

for (const contractCallEvent of entries) {
this.logger.debug(`Trying to verify ContractCallEvent with id ${contractCallEvent.id}`);
if (contractCallEvent.retry === MAX_NUMBER_OF_RETRIES) {
this.logger.error(
`Could not verify contract call event ${contractCallEvent.id} after ${contractCallEvent.retry} retries`,
);

await this.contractCallEventRepository.updateStatus(contractCallEvent.id, ContractCallEventStatus.FAILED);

try {
const response = await firstValueFrom(this.grpcService.verify(contractCallEvent));
continue;
}

if (!response.error) {
contractCallEvent.status = ContractCallEventStatus.APPROVED;
} else {
contractCallEvent.status = ContractCallEventStatus.FAILED;
contractCallEvent.retry += 1;

this.logger.error(
`Verify contract call event ${contractCallEvent.id} was not successful. Got error code ${response.error.errorCode}`,
);
}
} catch (e) {
this.logger.error(`Could not verify contract call event ${contractCallEvent.id}`);
this.logger.error(e);
this.logger.debug(`Trying to verify ContractCallEvent with id ${contractCallEvent.id}, retry ${contractCallEvent.retry}}`);

contractCallEvent.status = ContractCallEventStatus.FAILED;
}
await this.contractCallEventRepository.updateRetry(contractCallEvent.id, contractCallEvent.retry);

await this.contractCallEventRepository.updateStatus(contractCallEvent);
this.grpcService.verify(contractCallEvent);
}

page++;
Original file line number Diff line number Diff line change
@@ -6,14 +6,14 @@ import { NotifierBlockEvent } from './types';
import { GatewayProcessor, GasServiceProcessor } from '../processors';

describe('EventProcessorService', () => {
let contractCallProcessor: DeepMocked<GatewayProcessor>;
let gatewayProcessor: DeepMocked<GatewayProcessor>;
let gasServiceProcessor: DeepMocked<GasServiceProcessor>;
let apiConfigService: DeepMocked<ApiConfigService>;

let service: EventProcessorService;

beforeEach(async () => {
contractCallProcessor = createMock();
gatewayProcessor = createMock();
gasServiceProcessor = createMock();
apiConfigService = createMock();

@@ -25,7 +25,7 @@ describe('EventProcessorService', () => {
})
.useMocker((token) => {
if (token === GatewayProcessor) {
return contractCallProcessor;
return gatewayProcessor;
}

if (token === GasServiceProcessor) {
@@ -70,7 +70,7 @@ describe('EventProcessorService', () => {
await service.consumeEvents(blockEvent);

expect(apiConfigService.getContractGateway).toHaveBeenCalledTimes(1);
expect(contractCallProcessor.handleEvent).not.toHaveBeenCalled();
expect(gatewayProcessor.handleEvent).not.toHaveBeenCalled();
expect(gasServiceProcessor.handleEvent).not.toHaveBeenCalled();
});

@@ -93,7 +93,7 @@ describe('EventProcessorService', () => {
await service.consumeEvents(blockEvent);

expect(apiConfigService.getContractGateway).toHaveBeenCalledTimes(1);
expect(contractCallProcessor.handleEvent).toHaveBeenCalledTimes(1);
expect(gatewayProcessor.handleEvent).toHaveBeenCalledTimes(1);
expect(gasServiceProcessor.handleEvent).not.toHaveBeenCalled();
});

@@ -117,7 +117,7 @@ describe('EventProcessorService', () => {

expect(apiConfigService.getContractGateway).toHaveBeenCalledTimes(1);
expect(gasServiceProcessor.handleEvent).toHaveBeenCalledTimes(1);
expect(contractCallProcessor.handleEvent).not.toHaveBeenCalled();
expect(gatewayProcessor.handleEvent).not.toHaveBeenCalled();
});
});
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
export * from './message-approved.processor.module';
export * from './message-approved.processor.service';
Original file line number Diff line number Diff line change
@@ -2,10 +2,10 @@ import { Module } from '@nestjs/common';
import { ScheduleModule } from '@nestjs/schedule';
import { DatabaseModule } from '@mvx-monorepo/common';
import { ContractsModule } from '@mvx-monorepo/common/contracts/contracts.module';
import { CallContractApprovedProcessorService } from './call-contract-approved.processor.service';
import { MessageApprovedProcessorService } from './message-approved.processor.service';

@Module({
imports: [ScheduleModule.forRoot(), DatabaseModule, ContractsModule],
providers: [CallContractApprovedProcessorService],
providers: [MessageApprovedProcessorService],
})
export class CallContractApprovedProcessorModule {}
export class MessageApprovedProcessorModule {}
Loading