Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Implemented redis.asyncio.SentinelBlockingConnectionPool. #3321

Open
wants to merge 11 commits into
base: master
Choose a base branch
from

Conversation

DABND19
Copy link

@DABND19 DABND19 commented Jul 17, 2024

Pull Request check-list

Please make sure to review and check all of these items:

  • Do tests and lints pass with this change?
  • Do the CI tests pass with this change (enable it first in your forked repo and wait for the github action build to finish)?
  • Is the new or changed code fully tested?
  • Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?
  • Is there an example added to the examples folder (if applicable)?
  • Was the change added to CHANGES file?

NOTE: these things are not required to open a PR and can be done
afterwards / while the PR is open.

Description of change

Added redis.asyncio.SentinelBlockingConnectionPool class

@DABND19 DABND19 force-pushed the feature/sentinel-blocking-connection-pool branch from c8eecd5 to 713fa03 Compare July 17, 2024 14:14
@DABND19 DABND19 marked this pull request as draft July 18, 2024 11:59
@DABND19 DABND19 force-pushed the feature/sentinel-blocking-connection-pool branch from 713fa03 to baf2af2 Compare July 18, 2024 12:05
@DABND19 DABND19 force-pushed the feature/sentinel-blocking-connection-pool branch from baf2af2 to 9701492 Compare July 18, 2024 13:55
@DABND19 DABND19 changed the title feat: Implemented SentinelBlockingConnectionPool. feat: Implemented redis.asyncio.SentinelBlockingConnectionPool. Jul 18, 2024
@DABND19 DABND19 force-pushed the feature/sentinel-blocking-connection-pool branch 4 times, most recently from ed788bd to 6539da1 Compare July 18, 2024 14:15
@DABND19 DABND19 force-pushed the feature/sentinel-blocking-connection-pool branch 2 times, most recently from aca1d65 to f6168c3 Compare July 18, 2024 14:54
@DABND19 DABND19 force-pushed the feature/sentinel-blocking-connection-pool branch from f6168c3 to 9b1fbd6 Compare July 18, 2024 14:56
@DABND19 DABND19 marked this pull request as ready for review July 18, 2024 15:21
@DABND19
Copy link
Author

DABND19 commented Sep 1, 2024

@dvora-h Hello! How can I improve this pull request so that it gets reviewed?
And what's wrong with this pull request, which does some of the things I did in the current one, but at the same time it has been hanging around for a long time without approval?

@dvora-h
Copy link
Collaborator

dvora-h commented Sep 2, 2024

@vladvildanov

@@ -97,6 +107,55 @@ class SentinelManagedSSLConnection(SentinelManagedConnection, SSLConnection):
pass


class SentinelConnectionPoolProxy:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of this proxy class? get_master_address and rotate_slaves methods implementations are exactly the same as in SentinelConnectionPool class. The only difference that I see is the missing calls to parent object in constructor and reset.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did it this way because it is already done in the synchronous version: https://github.com/redis/redis-py/blob/master/redis/sentinel.py#L89

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I get it. It's a bad design approach and I don't want to spread it even more. It should be implemented correctly and I will create an issue for the future to do the same with sync version.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay. I'll think about a better solution

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The best solution will be refactor async ConnectionPool the way we don't need to use workarounds to break inheritance limitations

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello. What if we move BlockingConnectionPool logic to ConnectionPool?
We can do something like this:

from typing import Optional


# If the timeout parameter is zero, then we have the old behavior,
# else we have a blocking connection pool
class ConnectionPool:
    def __init__(
        self,
        timeout: Optional[int] = 0,
        **kwargs,
    ) -> None:
        ...


# For backward compatibility
class BlockingConnectionPool(ConnectionPool):
    def __init__(self, timeout: Optional[int] = 20, **kwargs) -> None:
        super().__init__(timeout=timeout, **kwargs)

@@ -116,12 +175,17 @@ def __init__(self, service_name, sentinel_manager, **kwargs):
)
self.is_master = kwargs.pop("is_master", True)
self.check_connection = kwargs.pop("check_connection", False)
self.proxy = SentinelConnectionPoolProxy(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, conceptually proxies are used to add an additional functionality before or after we want to access the original object. But here proxy are used just to override super class methods that breaks the functionality of child.

Proxy here is the workaround, to hide a problem with inheritance that we have. We have to either fix it in ConnectionPool so it can be inheritable, or doesn't inherit this at all

redis/asyncio/sentinel.py Show resolved Hide resolved
@cavemanpi
Copy link

@vladvildanov suggested I come over here as well. I have a PR which seems tangentially related. Any others care to give feedback on the idea expressed in this comment? #3359 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants