Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multi-threaded concurrent import question #4622

Open
dylan-tao opened this issue Aug 9, 2024 · 0 comments
Open

multi-threaded concurrent import question #4622

dylan-tao opened this issue Aug 9, 2024 · 0 comments

Comments

@dylan-tao
Copy link

Describe the feature:
Backend storage: Cassandra, multi-threaded concurrent import In addition to transactions and unique indexes to ensure data consistency, retry the import with transaction rollback, what other better way, the current performance is too low.

Describe a specific use case for the feature:
When importing data, enable multi-threaded concurrent writes, and the uniqueness of a node may have multiple inserts at the same time. The current solution is to start the transaction or commit or roll back as a whole. Execute the query first when inserting, and associate with the existing node if it exists, and insert if it does not exist. Since the node is diffuse derived, it is not possible to group and isolate the data through the associated node and then perform multi-threaded safe uniqueness import.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant