Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make merkle tree leaf addition operations atomic #71

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

jimthematrix
Copy link
Contributor

@jimthematrix jimthematrix commented Sep 17, 2024

fixes #21

Copy link
Contributor

@EnriqueL8 EnriqueL8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A view on the batch TX logic

// the node's reference key (not the index) is used as the key to
// store the node in the DB
n := core.SMTNode{
RefKey: ref.Hex(),
}
err := s.p.DB().Table(s.nodesTableName).First(&n).Error
err := batchOrDb.Table(nodesTableName).First(&n).Error
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm struggling why a read operation would need a batch transaction

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here we want the query to hit the records that have been added in the batch, but have not been committed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah gotcha

Copy link
Contributor

@EnriqueL8 EnriqueL8 Sep 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but if you are looking relying on getNode before inserting then you need to make sure to use the DB directly instead of batch

Comment on lines 281 to 284
if _, err := batch.GetNode(k); err == nil {
return nil, ErrNodeIndexAlreadyExists
}
err := mt.db.InsertNode(n)
err := batch.InsertNode(n)
Copy link
Contributor

@EnriqueL8 EnriqueL8 Sep 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is where I don't understand the batch function where you get a node but the TX is not committed yet and then you are adding an insert to the same transaction

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for example here, you would use db instead of batch for GetNode?

Copy link
Contributor

@EnriqueL8 EnriqueL8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarifications on the batching in GROM , makes sense to me

Copy link
Contributor

@EnriqueL8 EnriqueL8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After a conversation with @jimthematrix , we can move to ON CONFLICT insert and if there is on conflict do nothing and rework the way the read works

@jimthematrix
Copy link
Contributor Author

@EnriqueL8 thanks for catching that. just added a commit to make the inserts `ON CONFLICT DO NOTHING" with a justification in the inline comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sparse merkle tree implementation - hardening
2 participants