Skip to content
This repository has been archived by the owner on Aug 23, 2020. It is now read-only.

Feature: Transaction Solidifier #1484

Open
wants to merge 2 commits into
base: dev
Choose a base branch
from

Conversation

luca-moser
Copy link
Member

@luca-moser luca-moser commented Jun 5, 2019

Description

In order to have a relevant view of the graph from a given depth towards the tips, we want to be able to mark transactions as solid from bottom->top. In a scenario where the TipSolidifier is disabled, only the TransactionValidator will mark transactions as solid (besides the MilestoneSolidifer), by performing once a quick solid check on approvers of transactions which were set to be solid up on arrival. The TransactionsValidator's quick solidification thread runs every 750 milliseconds.

This means that if we are receiving transactions out of order and a given transaction was missed by the TransactionValidator quick solidification thread, the transaction will only be marked as solid once a new milestone is issued (because it triggers a solidification of all approved transactions).

This PR adds a separate component called TransactionSolidifier with its implementation QuickTransactionSolidifier which starts at a configurable depth (default: 3) and walks from the given milestone up the graph towards the tips traversing all transaction and performing a quick solid check on each of them. The QuickTransactionSolidifier runs every 10 seconds (configurable).

Testing this component on the ICC network, I wanted to see how long such mechanism would take. Luckily, the operation seems to be very quick as shown by the given log outputs:

06/05 17:49:30.697 [Quick Transaction Solidifier] INFO  QuickTransactionSolidifier:88 - updated 19 and traversed 1321 transactions, took 45 ms
06/05 17:49:33.357 [Tips Requester] INFO  TipsRequesterImpl:89 - toProcess = 0 , toBroadcast = 0 , toRequest = 0 , toReply = 0 / totalTransactions = 3251301
06/05 17:49:40.763 [Quick Transaction Solidifier] INFO  QuickTransactionSolidifier:88 - updated 21 and traversed 1385 transactions, took 65 ms
06/05 17:49:43.358 [Tips Requester] INFO  TipsRequesterImpl:89 - toProcess = 0 , toBroadcast = 0 , toRequest = 0 , toReply = 0 / totalTransactions = 3251365
06/05 17:49:50.846 [Quick Transaction Solidifier] INFO  QuickTransactionSolidifier:88 - updated 9 and traversed 1439 transactions, took 77 ms
06/05 17:49:53.360 [Neighbor Router] INFO  NeighborRouter:551 - establishing connections to 1 wanted neighbors [tcp://node03.x-vps.com:15600]
06/05 17:49:53.360 [Tips Requester] INFO  TipsRequesterImpl:89 - toProcess = 0 , toBroadcast = 0 , toRequest = 0 , toReply = 0 / totalTransactions = 3251413
06/05 17:50:00.898 [Quick Transaction Solidifier] INFO  QuickTransactionSolidifier:88 - updated 7 and traversed 1490 transactions, took 51 ms
06/05 17:50:03.361 [Tips Requester] INFO  TipsRequesterImpl:89 - toProcess = 0 , toBroadcast = 0 , toRequest = 0 , toReply = 0 / totalTransactions = 3251461
06/05 17:50:04.456 [Neighbor Router] INFO  NeighborRouter:275 - couldn't establish connection to neighbor 116.203.156.13:15600, will attempt to reconnect later. reason: Connection timed out
06/05 17:50:10.962 [Quick Transaction Solidifier] INFO  QuickTransactionSolidifier:88 - updated 21 and traversed 1536 transactions, took 63 ms
06/05 17:50:13.363 [Tips Requester] INFO  TipsRequesterImpl:89 - toProcess = 0 , toBroadcast = 0 , toRequest = 0 , toReply = 0 / totalTransactions = 3251507
06/05 17:50:21.023 [Quick Transaction Solidifier] INFO  QuickTransactionSolidifier:88 - updated 17 and traversed 1582 transactions, took 60 ms
06/05 17:50:23.366 [Tips Requester] INFO  TipsRequesterImpl:89 - toProcess = 0 , toBroadcast = 0 , toRequest = 0 , toReply = 0 / totalTransactions = 3251551
06/05 17:50:31.078 [Quick Transaction Solidifier] INFO  QuickTransactionSolidifier:88 - updated 9 and traversed 1624 transactions, took 54 ms

Note that updated N means that the N transactions were updated to be solid and traversed N means how many transactions were traversed while walking up the graph towards the tips. Note also that updated N shows that the TransactionValidator quick solidification thread misses some transactions when it runs which are however always going to be catched by the QuickTransactionSolidifier.

This component doesn't appear even in YourKit's profiling because it's very cheap.

Fixes # (issue)
#1009
#1013
#1011 (maybe?)

The solidifier in TransactionValidator can be removed through #1485.

Type of change

  • Enhancement (a non-breaking change which adds functionality)

How Has This Been Tested?

On the ICC network

Checklist:

Please delete items that are not relevant.

  • My code follows the style guidelines for this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • New and existing unit tests pass locally with my changes

@kwek20
Copy link
Contributor

kwek20 commented Aug 31, 2019

Some unit tests should be made for this

Copy link
Contributor

@GalRogozinski GalRogozinski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took some time this morning to finally look at this.

Looks pretty good. If you will fix the conflicts and the few comments then I will get it merged quickly



/**
* The depth the solidifier will use.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add Default Value like in the other configs

int latestMilestoneIndex = latestMilestoneTracker.getLatestMilestoneIndex();
int depth = solidificationConfig.getSolidifierDepth();
if (latestMilestoneIndex - depth <= 0
|| snapshotProvider.getLatestSnapshot().getIndex() < latestMilestoneIndex - depth) {
Copy link
Contributor

@GalRogozinski GalRogozinski Sep 4, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

snapshotProvider.getLatestSnapshot().getIndex() < latestMilestoneIndex - depth
Maybe I am getting confused but doesn't this contradict your comment?

I read as if you are not synchronized then don't attempt to solidify

int getSolidifierDepth();

/**
* Returns the interval at which the solidifer will run.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add Default Value like in the other configs

@iotaledger iotaledger deleted a comment Sep 30, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants