-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[IO] Add Redis Connector #289
base: experimental
Are you sure you want to change the base?
Conversation
OMG!!! I merged these commits for several hours... but they come out again... maybe I made a ridiculous mistake that override the merge operations.... OMG.... actually, "[IO] Add Redis Connector. Fixed merge conflicts." (38ce3c2) is supposed to be the ONLY ONE left. I am currently searching for safely merging solutions. |
f322ea9
to
95228f4
Compare
Hi Legend, |
This PR is including other commits in master branch which are not merged into experimental branch. Could you redo the rebase? |
No problem. |
d0566fa
to
1ecd4b8
Compare
This branch is updated to master now. Please rebase again! |
And you can run |
Test cases
- Preparation
- Test 1: CMake
Expectation: running well.
Notes:
- Test 2: enwiki from MongoDB to Redis:
Expectation: running well. 4825150 records to be imported. around 1m30s.
Notes: MUST wait for 10 minutes before Test 3, because for the first time data were loaded into Redis cluster, it takes time for master-slave synchronization.
- Test 3: enwiki from Redis to Redis:
Expectation: running well. 4825150 records to be imported. around 1m.
Notes: On Husky-worker side, "waiting for keys" logs may be printed as expected, which means that some workers are waiting for keys from the next batch.
- Test 4: enwiki WordCount:
Expectation: running well. 4825150 records to be imported. around 50s.
Notes: Can be performed consecutively without restart Husky-Master.