Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Critical bug fix: Fix cluster-mode TLS disabled client to properly take received IPs when DNS is expanded #173

Merged
merged 1 commit into from
Jul 11, 2024

Conversation

barshaul
Copy link

@barshaul barshaul commented Jul 11, 2024

Bug Description

This bug affects clusters in non-TLS setups that use a DNS endpoint storing multiple addresses, such as the cluster discovery endpoint in ElastiCache/MemoryDB.

Issue

When expanding a DNS endpoint from the initial nodes, we store the socket_address received from the expansion. However, in non-TCP connections, we haven't used the received socket_address. Instead, we re-resolved the original DNS entry when creating the actual connection. This caused a faulty mapping between the node address and the actual connection.

Example Scenario

Consider the endpoint cluster.dns.endpoint expanded to IP1, IP2, and IP3. Previously, all nodes were mapped to a connection with the first address resolved from cluster.dns.endpoint. If a DNS lookup for cluster.dns.endpoint returns the order [IP1, IP2, IP3], the connection map would be:

  • IP1 -> connection to IP1
  • IP2 -> connection to IP1
  • IP3 -> connection to IP1

As a result, requests routed to IP2 and IP3 were actually sent to IP1, leading to MOVED errors.

Fix

This fix modifies the connect_simple function to use the provided socket_address, if available, ensuring that connections are made to the addresses used as keys in the connection map. This adjustment aligns the non-TCP connection behavior with that of the TcpTls connection.

Follow-Up

We should mock a DNS entry or the dnslookup function and add relevant tests to validate this fix.

@barshaul barshaul requested review from avifenesh and removed request for avifenesh July 11, 2024 12:35
@barshaul barshaul merged commit ee3119d into amazon-contributing:main Jul 11, 2024
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants