-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature] Support the Arrow Flight SQL protocol to enhance data transfer efficiency #50199
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: liubotao <[email protected]>
Signed-off-by: liubotao <[email protected]>
fe/fe-core/src/main/java/com/starrocks/service/arrow/flight/sql/ArrowFlightSqlServiceImpl.java
Show resolved
Hide resolved
...ore/src/main/java/com/starrocks/service/arrow/flight/sql/ArrowFlightSqlConnectProcessor.java
Show resolved
Hide resolved
would better give the performance report |
@liubotao |
...main/java/com/starrocks/service/arrow/flight/sql/auth/ArrowFlightSqlCredentialValidator.java
Outdated
Show resolved
Hide resolved
Signed-off-by: liubotao <[email protected]>
Signed-off-by: liubotao <[email protected]>
Related Issue:#50285 |
Signed-off-by: liubotao <[email protected]>
Signed-off-by: liubotao <[email protected]>
Signed-off-by: liubotao <[email protected]>
Signed-off-by: liubotao <[email protected]>
Signed-off-by: liubotao <[email protected]>
3f467b3
to
72bdeac
Compare
Signed-off-by: liubotao <[email protected]>
Signed-off-by: liubotao <[email protected]>
Signed-off-by: liubotao <[email protected]>
Signed-off-by: liubotao <[email protected]>
Signed-off-by: liubotao <[email protected]>
Signed-off-by: liubotao <[email protected]>
Signed-off-by: liubotao <[email protected]>
[BE Incremental Coverage Report]❌ fail : 0 / 130 (00.00%) file detail
|
[FE Incremental Coverage Report]❌ fail : 72 / 553 (13.02%) file detail
|
Signed-off-by: liubotao <[email protected]>
Quality Gate failedFailed conditions See analysis details on SonarCloud Catch issues before they fail your Quality Gate with our IDE extension SonarLint |
[Java-Extensions Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
Why I'm doing:
Significantly enhance data transfer efficiency when using StarRocks for data science computations.
What I'm doing:
This PR adds support for Apache Arrow Flight SQL to StarRocks. Arrow Flight is a high-performance remote procedure call (RPC) framework specifically designed for large-scale data set transmission. By integrating Arrow Flight SQL, StarRocks can achieve more efficient data transfer and query execution, with expected data transfer speed improvements of up to 10 to 100 times. Additionally, StarRocks' Arrow Flight SQL service supports connections through the Python ADBC (Arrow Database Connectivity) standard interface and JDBC (Java Database Connectivity) drivers. This allows significant acceleration of queries and data transfers without needing to modify existing SQL statements, thereby greatly enhancing data analysis performance and user experience
Fix: #50285
What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist:
Bugfix cherry-pick branch check: