Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Command Migration: ('INCR', 'INCRBY', 'INCRBYFLOAT', 'DECR', 'DECRBY') #1016

Closed
4 tasks
AshwinKul28 opened this issue Oct 8, 2024 · 9 comments · Fixed by #1141
Closed
4 tasks

Command Migration: ('INCR', 'INCRBY', 'INCRBYFLOAT', 'DECR', 'DECRBY') #1016

AshwinKul28 opened this issue Oct 8, 2024 · 9 comments · Fixed by #1141
Assignees
Labels
migration -- command Migrates current eval to a refactored eval for all protocols functionality

Comments

@AshwinKul28
Copy link
Contributor

This issue tracks the migration of the mentioned commands - ('INCR', 'INCRBY', 'INCRBYFLOAT', 'DECR', 'DECRBY') to make them compatible across the three protocols supported by the Dice database: RESP, HTTP, and WebSocket.
DiceDB now supports more than one protocols (Resp/http/websocket). We don't want eval operation of each command to be strictly bound with any of the protocols. Currently eval function return statements of each command is bounded to the RESP protocol.

The migration is required because:

  • The current implementation is specific to the RESP protocol
  • Evaluation functions of each command must be refactored to ensure protocol-agnostic behavior.
  • Evaluation functions should return raw responses instead of protocol specific responses.

The goal is to make the command logic protocol-independent, allowing all three protocols to call the same core functionality seamlessly.

Requirements

  • Refactor the evaluation function of [Command Name] to be generic.
  • Ensure the evaluation function does not include protocol-specific logic.

Migration Steps

  1. Analyze Current Implementation

    • Review the current codebase to understand how the command logic is implemented.
  2. Refactor Return Logic

    • Create a new function with the function definition as evalXXX(args []string, store *dstore.Store) *EvalResponse under the file /internal/eval/store_eval.go
    • Analyse return statements of the eval and modify them to send raw types without encoding
    • Use errors in the return statements from /errors/migrated_errors.go file
    • Use perdefined responses from /internal/clientio/resp.go file as RespType variables
  3. Command/Worker specific Changes

    • Make IsMigrated flag to true in the commands information under /internal/eval/commands.go file.
    • Use the newly written eval function against the NewEval parameter in the same command structure.
    • Delete the old eval function from the /internal/eval/eval.go file.
    • Add the migrated command to the /internal/worker/CommandsMeta map and make it's type as SingleShard
  4. Update Unit Tests

    • Refactor existing unit tests to accommodate the new implementation.
    • Add new unit tests if necessary to cover all possible cases.
  5. Integration Tests

    • Run all integration tests to ensure successful migration.
    • Ensure that each protocol (RESP, HTTP, WebSocket) works correctly after migration.

Checklist

  • Migrated the evalXXX function with the latest definition
  • Update or add unit tests for the new implementation.
  • All unit tests pass successfully.
  • Ensure all integration tests pass successfully.

Additional Notes

  • Describe any edge cases that need to be handled in the generic function.
  • Mention any protocol-specific optimizations required in wrappers.

If there are any questions or concerns about this migration, please mention them here.

Related Issues/PRs

  • Sample implementation for the Get, Set, GetSet and SetEx can be found in this Pull Request.
@pg30
Copy link
Contributor

pg30 commented Oct 8, 2024

@AshwinKul28 I would like to give this a try, Can you pls assign this to me.

@anchalsinghrajput
Copy link

Hi @AshwinKul28, can I pick this up?

@AshwinKul28
Copy link
Contributor Author

HI @pg30 go for it. Thanks for picking this up. 🚀

@anchalsinghrajput Thanks for your interest. please pick other opened issues. There are many for command migration.

@pg30
Copy link
Contributor

pg30 commented Oct 11, 2024

In the old implementation of INCR command :

  1. If the object is of type SET it throws the wrong type error but if its any other type like HMAP it throws : (error) ERR value is not an integer or out of range. It should throw the same error for all the types other than INT?
  2. Checked the same behaviour in redis and it correctly throws the wrong type error for the type inconsistencies.

@pg30
Copy link
Contributor

pg30 commented Oct 12, 2024

I am not able to find the unit tests for the existing implementations of INCR, INCRBY, DECR, DECRBY and thus will be adding all of those in the same PR.

@AshwinKul28
Copy link
Contributor Author

HI @pg30 thanks for the updates. Yes please make it as close as Redis, also go ahead and add unit test cases. Although I can see few of them are covered in the eval_test.go file.

@pg30
Copy link
Contributor

pg30 commented Oct 13, 2024

Hey @AshwinKul28 , Just a suggestion
Currently all the eval test cases have a generic test runner in the function runEvalTests but we don't have anything for the migration. Already migrated commands are writing their own test runners testEvalSET, testEvalGET and are doing mostly the same thing by duplicating the code. Do you think we should create a generic test runner for this as well incorporating setup and validator call ?

@soumya-codes
Copy link
Contributor

@pg30 hope you are doing well. Any updates on the progress? We are targeting these issues little aggressively as we maintaining the commands across two separate server implementations is getting very cumbersome.

@soumya-codes soumya-codes added the migration -- command Migrates current eval to a refactored eval for all protocols functionality label Oct 17, 2024
@pg30
Copy link
Contributor

pg30 commented Oct 17, 2024

Hii,
Really sorry for the delay as I was travelling for the past 2 days and couldn't work on it.
Though I have migrated all the commands and written the unit tests as there weren't any existing. My integration tests are failing probably due to some setup issue as they are unrelated to my work.
Will try to resolve it by tomorrow.
Thanks for checking in.

AshwinKul28 pushed a commit that referenced this issue Oct 20, 2024
* migrated incr commands

* fixed test cases

* fixed integration tests

* fixed linting issues

* updated docs and added error for consistency with redis

* fixed integration tests

* fixed

* updated docs

* integration tests

* update

* integration tests

* added integration tests

* tests

* fix

* added integration tests for websocket

* updated docs:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
migration -- command Migrates current eval to a refactored eval for all protocols functionality
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants