Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Chainnotifier Service to leverage batch-request support #1

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

Vib-UX
Copy link
Owner

@Vib-UX Vib-UX commented Apr 13, 2022

BatchRPC

Copy link

@guggero guggero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good! Added a few ideas for optimizations.

chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
@Vib-UX Vib-UX changed the title Update historicalSpendDetails Update Chainnotifier Service to leverage batch-request support Apr 18, 2022
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
go.mod Outdated Show resolved Hide resolved
chainntnfs/btcdnotify/btcd.go Outdated Show resolved Hide resolved
chainntnfs/btcdnotify/btcd.go Outdated Show resolved Hide resolved
@Vib-UX Vib-UX force-pushed the BatchRPC branch 2 times, most recently from 8593d86 to 2461109 Compare July 13, 2022 16:55
Copy link

@bhandras bhandras left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome progress, getting very close! Great work @Vib-UX :)

chainntnfs/btcdnotify/btcd.go Outdated Show resolved Hide resolved
DisableConnectOnNew: true,
Certificates: config.Certificates,
Endpoint: config.Endpoint,
HTTPPostMode: true,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure but I don't think btcd requires http post mode?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

When working with updated batch client HTTPPostMode is required.

chainntnfs/btcdnotify/btcd.go Outdated Show resolved Hide resolved
chainntnfs/btcdnotify/btcd.go Outdated Show resolved Hide resolved
chainntnfs/btcdnotify/btcd.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Show resolved Hide resolved
chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
@@ -221,7 +222,7 @@ func testHistoricalConfDetailsNoTxIndex(t *testing.T, rpcpolling bool) {
unknownConfReq, err := chainntnfs.NewConfRequest(&unknownHash, testScript)
require.NoError(t, err, "unable to create conf request")
broadcastHeight := syncNotifierWithMiner(t, notifier, miner)
_, txStatus, err := notifier.historicalConfDetails(
txConfirmation, txStatus, err := notifier.historicalConfDetails(
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getting txConfirmation here will increase the coverage by verifying the output results.

@bhandras
Copy link

bhandras commented Aug 3, 2022

This is the latest version @Vib-UX ?

@Vib-UX
Copy link
Owner Author

Vib-UX commented Aug 3, 2022

It's not the updated one, need to change the replace directive for the updated btcwallet and some nit fix, will push updates by tomm eod.

@Vib-UX Vib-UX force-pushed the BatchRPC branch 2 times, most recently from 36dbc33 to e9cc98f Compare August 6, 2022 06:46
Copy link

@bhandras bhandras left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I think the indexing is off. Could be I'm wrong which also means some extra comments could be added :)

🚀

return &chainntnfs.TxConfirmation{
Tx: tx,
BlockHash: blockHashes[start+j],
BlockHeight: uint32(height - j),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is also a + ? Maybe I'm wrong, but since we iterate from currentHeight backwards, height is always the start of the current batch iuuc.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be -ve only, as we traverse height from currentHeight to heightHint with height--. Now since we are traversing backwards.

Lets take an example,

  • currentHeight = 100
  • start=0
  • batchSize = 15
  • j is traversing from block=100 to block=86

So here if we found the relevantTxMatch at j=0 we return currentHeight-j --> 100-0 = 100

return nil, chainntnfs.TxNotFoundManually, err
}

height := int(currentHeight) - start
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps worth adding a comment here to explain what we track with height.

SpenderTxHash: &txHash,
SpendingTx: tx,
SpenderInputIndex: inputIdx,
SpendingHeight: int32(height - j),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, I believe this should be +, see other comment.

}
if !matches {
continue
height := int(endHeight) - start
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add some comment there to help readers understand the stepping.

chainntnfs/bitcoindnotify/bitcoind.go Outdated Show resolved Hide resolved
@bhandras
Copy link

bhandras commented Aug 8, 2022

Ready to open against upstream once all comments are addressed. 🎉

go.mod Outdated Show resolved Hide resolved
require.Equal(t, validSpendDetails.SpentOutPoint, outpoint)
require.Equal(t, validSpendDetails.SpenderTxHash, spendTxHash)
require.Equal(t, validSpendDetails.SpendingTx, spendTx)
require.Equal(t, validSpendDetails.SpendingHeight, int32(broadcastHeight+1))
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if indexing would be off this would have thrown error.

I compared actualSpendHeight with received BlockHeight from validSpendDetails.

With this RegisterSpendNtfn() & RegisterConfirmationsNtfn() services
will leverage the support of updated batchAPI for bitcoind making it
lightning fast.
Testcase to ensure the updated historicalSpendDetails() and confDetailsManually()
which leverages batchAPI for scanning the chain manually works as expected
with enhanced performance.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants