Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(store): fix inf.Dec and big.Int compare #406

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

illia-li
Copy link
Contributor

@illia-li illia-li commented Sep 12, 2023

As solution #405 (comment)
Main changes:

  1. Responses from stores compare as []bytes, instead of values unmarshalled by "gocql" .This eliminates issue 'varint' and 'decimal' scylla types does`t work properly as a maps key. #405.
  2. Custom difference information collector, instead of cmp.Diff. Now information collect more user friendly and can be customized.

'gemini' productivity has increased on 30% for responses with big rows count and on 8% for responses with usual rows count.

@illia-li illia-li force-pushed the il/fix-store-compare_map_bigintv3 branch from c3e5c7b to 0dd9953 Compare September 12, 2023 21:56
@illia-li illia-li force-pushed the il/fix-store-compare_map_bigintv3 branch 4 times, most recently from 2e0a3de to d04ca41 Compare December 9, 2023 23:34
@@ -0,0 +1,123 @@
// Copyright 2019 ScyllaDB
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copyright 2023-present

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -0,0 +1,137 @@
// Copyright 2019 ScyllaDB
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copyright 2023-present

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -0,0 +1,288 @@
// Copyright 2019 ScyllaDB
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copyright 2023-present

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -90,11 +92,25 @@ func (cs *cqlStore) doMutate(ctx context.Context, builder qb.Builder, ts time.Ti
return nil
}

func (cs *cqlStore) load(ctx context.Context, builder qb.Builder, values []interface{}) (result []map[string]interface{}, err error) {
func (cs *cqlStore) loadSV(ctx context.Context, builder qb.Builder, values []interface{}) (sv.Result, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SV and MV are cryptic.
Why not expand them here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expanded

@@ -0,0 +1,198 @@
// Copyright 2019 ScyllaDB
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copyright 2023-present

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

out += fmt.Sprintf("%d:%s;", idx, l[idx].ToString(listInfo.Elem))
}
out = out[:len(out)-1]
return out + ">"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you modify the last char to > in place instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't. It's string, simple chars modifications impossible. Modified to return out[:len(out)-1] + ">"

changes in check of responses rows procces:
*rows data unmarshalling into byte slices.
*rows data compare as byte slices, without transformation to 'GO' types.
*rows diff information now more user friendly and can be customed. Diff collect row by row to string slice and can be transferred anywhere with better readable.
2) remake ColumnRaw to string instead []byte
@illia-li illia-li force-pushed the il/fix-store-compare_map_bigintv3 branch from 49104c5 to 2fbfc33 Compare December 21, 2023 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants