Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shallow Clone #7486

Merged
merged 19 commits into from
Feb 16, 2024
Merged

Shallow Clone #7486

merged 19 commits into from
Feb 16, 2024

Conversation

macneale4
Copy link
Contributor

@macneale4 macneale4 commented Feb 13, 2024

Add the --depth flag to the dolt_clone stored procedure and the dolt cli

This is possibly an MVP, as it's kind of impossible to test everything this touches. The guiding principle was to not break the code for any fully cloned repository.

There currently remain 7 problems to address in this code. They are each marked with "NM4"

@coffeegoddd

This comment was marked as outdated.

@coffeegoddd

This comment was marked as outdated.

@coffeegoddd

This comment was marked as outdated.

@coffeegoddd

This comment was marked as outdated.

@coffeegoddd

This comment was marked as outdated.

@coffeegoddd
Copy link
Contributor

@macneale4 DOLT

comparing_percentages
99.999562 to 99.999562
version result total
ddf74aa not ok 26
ddf74aa ok 5937431
version total_tests
ddf74aa 5937457
correctness_percentage
99.999562

Copy link
Contributor

@reltuk reltuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Took a first pass at comments. Nothing glaring, generally looks great :).

go/libraries/doltcore/doltdb/commit.go Show resolved Hide resolved
@@ -142,7 +184,15 @@ func GetCommitAncestor(ctx context.Context, cm1, cm2 *Commit) (*Commit, error) {
return nil, err
}

return NewCommit(ctx, cm1.vrw, cm1.ns, targetCommit)
if targetCommit.IsGhost() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we expecting this to return ErrNoCommonAncestor in the case where we don't find the common ancestor because it's a ghost commit? Or does this actually work because the common ancestor logic is against the commit closure, which we have?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning the ErrNoCommonAncestor would result in the wrong error. The way it is now (and we have a merge-base test for this), we message to the user that their shallow clone is incomplete and they should do a full clone.

go/libraries/doltcore/doltdb/commit.go Show resolved Hide resolved
go/libraries/doltcore/env/actions/remotes.go Outdated Show resolved Hide resolved
go/libraries/doltcore/env/actions/remotes.go Outdated Show resolved Hide resolved
go/store/nbs/generational_chunk_store.go Show resolved Hide resolved
go/store/nbs/generational_chunk_store.go Outdated Show resolved Hide resolved
go/store/nbs/generational_chunk_store.go Outdated Show resolved Hide resolved
go/store/nbs/generational_chunk_store.go Outdated Show resolved Hide resolved
return nil
}

func (g *GhostBlockStore) PersistGhostHashes(ctx context.Context, hashes hash.HashSet) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No locking around the shared state needed here?

Should this method error if skippedRefs is not empty?

Would it be slightly more robust to write the file to a different, temporary filename and only rename it to g.ghostObjectsFile if the file contents successfully land on disk?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the updates aren't racy. Did add a check to ensure we never write an empty set.

@coffeegoddd
Copy link
Contributor

@macneale4 DOLT

comparing_percentages
99.999562 to 99.999562
version result total
050a258 not ok 26
050a258 ok 5937431
version total_tests
050a258 5937457
correctness_percentage
99.999562

@macneale4 macneale4 merged commit 139c49f into main Feb 16, 2024
19 checks passed
@macneale4 macneale4 deleted the macneale4/shallow-clone branch May 30, 2024 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants