Skip to content

Custom proto revision #1553

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

titanproger
Copy link

@titanproger titanproger commented May 7, 2025

Summary

Add ability to setup in options preferable Native TCP protocol version, to optimise data size, downloaded from server.

#1552

Checklist

Delete items not relevant to your PR:

  • Unit and integration tests covering the common scenarios were added
  • A human-readable description of the changes was provided to include in CHANGELOG - comment: not sure should i change CHANGELOG without version changing - it looks like generated automatically by some script
  • For significant changes, documentation in https://github.com/ClickHouse/clickhouse-docs was updated with further explanations or tutorials - comment: not a significant change

@CLAassistant
Copy link

CLAassistant commented May 7, 2025

CLA assistant check
All committers have signed the CLA.

@titanproger titanproger force-pushed the custom-proto-revision branch 2 times, most recently from e6186a3 to d906adf Compare May 7, 2025 19:50
@titanproger
Copy link
Author

As a justification that the library is supporting different revisions of protocol:
here revision can be downgraded from server side

@titanproger titanproger force-pushed the custom-proto-revision branch from d906adf to 7a195fb Compare May 9, 2025 12:08
@mshustov mshustov requested review from Copilot and SpencerTorres May 13, 2025 15:53
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces support for a custom protocol revision for Native TCP to optimize data transfer size. Key changes include:

  • Added tests in tests/std/conn_test.go for verifying custom and unsupported protocol revisions.
  • Updated handshake, query sending, and connection dialing logic to use a dynamic protocol revision.
  • Extended Options in clickhouse_options.go to support custom ClientTCPProtocolVersion with appropriate default handling.

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/std/conn_test.go Added new tests to validate protocol revision behavior.
lib/proto/handshake.go Modified Decode method to incorporate clientRevision.
conn_send_query.go Updated query construction to use the dynamic protocol revision.
conn_handshake.go Updated handshake calls to pass dynamic protocol revision.
conn.go Added validation for the provided protocol revision and updated connection revision usage.
clickhouse_options.go Extended options to include ClientTCPProtocolVersion with a default value.
Comments suppressed due to low confidence (3)

conn.go:94

  • [nitpick] Consider including the provided ClientTCPProtocolVersion value in the error message for improved debugging clarity.
if opt.ClientTCPProtocolVersion < proto.DBMS_MIN_REVISION_WITH_CLIENT_INFO || opt.ClientTCPProtocolVersion > proto.DBMS_TCP_PROTOCOL_VERSION {

clickhouse_options.go:398

  • [nitpick] Using the same identifier for both the option field and the default constant may be confusing; consider renaming one for improved clarity.
if o.ClientTCPProtocolVersion == 0 { o.ClientTCPProtocolVersion = ClientTCPProtocolVersion }

lib/proto/handshake.go:101

  • Ensure that the 'min' function is defined or imported so that this code does not lead to runtime errors.
rev := min(clientRevision, srv.Revision)

@SpencerTorres
Copy link
Member

Hey! I reached out to your team in Slack to see if we can get some more details about your use case. But until that channel is available I'll make some notes here:

Why would you want to change the protocol version without also adding the features in the code? I fear this will break client compatibility. If it were safe to override the client revision, why can't we simply update the client revision?

I followed the link in the discussion as well, the linked line didn't seem relevant for what you were referring to. I think I need more context. I don't want to expose your implementation details on a public PR so feel free to reach out in the Slack channel.

Thanks! 😄

@titanproger
Copy link
Author

titanproger commented May 15, 2025

Hey! I reached out to your team in Slack to see if we can get some more details about your use case. But until that channel is available I'll make some notes here:

Why would you want to change the protocol version without also adding the features in the code? I fear this will break client compatibility. If it were safe to override the client revision, why can't we simply update the client revision?

I followed the link in the discussion as well, the linked line didn't seem relevant for what you were referring to. I think I need more context. I don't want to expose your implementation details on a public PR so feel free to reach out in the Slack channel.

Thanks! 😄

Thanks for feedback, @SpencerTorres !
let me provide some context:

Why would you want to change the protocol version without also adding the features in the code?

It is not allowed to set protocol version bigger than actually supported by client, or server
It is all about decrease protocol version and disabling unwanted features.

I fear this will break client compatibility.

it should not - as, by default, if not specify desire proto version in options, it will use latest protocol version - as it was before this PR

If it were safe to override the client revision, why can't we simply update the client revision?

it is all about downgrade version, not upgrade version. Downgrade client should be avoided , also there is no 2.* version of client that uses minimal protocol version.

I followed the link in the discussion as well, the linked line didn't seem relevant for what you were referring to. I think I need more context.

Basically, the reason why i want to be able downgrade protocol version, is to reduce unwanted metadata downloaded from server. And, i think, if setup version protocol is available for ch-go library users, it will be good to also support it in clickhouse-go client. (by the way maybe it is good idea to name this option with same name - like ProtocolVersion, or NativeProtocolVersion to emphasise it for native proto only)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants