Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](jdbc catalog) Get statistics using remote name #46040

Merged
merged 2 commits into from
Dec 27, 2024
Merged

[fix](jdbc catalog) Get statistics using remote name #46040

merged 2 commits into from
Dec 27, 2024

Conversation

zy-kkk
Copy link
Member

@zy-kkk zy-kkk commented Dec 26, 2024

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #41510

Problem Summary:

We should use remote name to get statistics

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Dec 26, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zy-kkk
Copy link
Member Author

zy-kkk commented Dec 26, 2024

run buildall

Jibing-Li
Jibing-Li previously approved these changes Dec 26, 2024
View reviewed changes
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 26, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 32775 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ef56db55fbb6b553428b0d9fdefdbd5df08d3e60, data reload: false

------ Round 1 ----------------------------------
q1	17601	6198	6013	6013
q2	2080	312	174	174
q3	10394	1283	725	725
q4	10227	866	451	451
q5	7522	2222	1969	1969
q6	214	179	150	150
q7	911	768	621	621
q8	9228	1367	1214	1214
q9	5333	4852	4957	4852
q10	6770	2303	1848	1848
q11	477	280	275	275
q12	356	369	224	224
q13	17786	3599	2983	2983
q14	231	227	214	214
q15	542	519	506	506
q16	640	626	601	601
q17	580	851	328	328
q18	7021	6474	6523	6474
q19	1375	962	569	569
q20	318	335	197	197
q21	2913	2228	2072	2072
q22	370	328	315	315
Total cold run time: 102889 ms
Total hot run time: 32775 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6236	6229	6204	6204
q2	245	327	230	230
q3	2264	2649	2365	2365
q4	1459	1809	1364	1364
q5	4371	4797	4793	4793
q6	191	185	148	148
q7	2109	1975	1834	1834
q8	2600	2775	2667	2667
q9	7250	7194	7178	7178
q10	3119	3297	2852	2852
q11	592	521	510	510
q12	672	731	650	650
q13	3375	3785	3124	3124
q14	277	320	306	306
q15	556	505	504	504
q16	651	694	653	653
q17	1239	1731	1258	1258
q18	7816	7382	7265	7265
q19	844	1069	1125	1069
q20	1925	1955	1915	1915
q21	5486	5269	4869	4869
q22	598	645	582	582
Total cold run time: 53875 ms
Total hot run time: 52340 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190849 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ef56db55fbb6b553428b0d9fdefdbd5df08d3e60, data reload: false

query1	979	391	399	391
query2	6515	2327	2221	2221
query3	6715	206	210	206
query4	34094	24370	23690	23690
query5	4373	660	470	470
query6	291	207	183	183
query7	4625	516	308	308
query8	305	241	227	227
query9	9633	2758	2746	2746
query10	468	324	254	254
query11	18173	15637	15309	15309
query12	154	107	111	107
query13	1668	568	418	418
query14	9916	7548	6822	6822
query15	243	207	192	192
query16	8333	612	444	444
query17	1592	769	592	592
query18	2120	420	306	306
query19	216	193	162	162
query20	120	121	114	114
query21	220	125	110	110
query22	4466	4420	4146	4146
query23	34284	33477	33420	33420
query24	6509	2316	2267	2267
query25	501	486	421	421
query26	1196	277	159	159
query27	2042	473	332	332
query28	5315	2490	2476	2476
query29	675	561	410	410
query30	235	184	158	158
query31	992	896	828	828
query32	87	61	59	59
query33	528	361	302	302
query34	787	882	532	532
query35	806	811	756	756
query36	1026	1044	973	973
query37	115	103	78	78
query38	4127	4311	4344	4311
query39	1505	1463	1453	1453
query40	211	114	104	104
query41	45	46	48	46
query42	119	104	102	102
query43	509	522	486	486
query44	1357	808	818	808
query45	181	171	171	171
query46	918	1080	666	666
query47	1881	1926	1899	1899
query48	391	415	319	319
query49	778	474	396	396
query50	646	693	395	395
query51	7166	7295	6981	6981
query52	103	102	90	90
query53	228	264	197	197
query54	491	499	409	409
query55	80	76	82	76
query56	253	275	235	235
query57	1203	1199	1147	1147
query58	249	232	233	232
query59	3133	3293	3001	3001
query60	282	270	256	256
query61	114	111	115	111
query62	878	794	725	725
query63	243	193	200	193
query64	4447	1000	658	658
query65	3242	3204	3218	3204
query66	1087	422	308	308
query67	16023	15882	15722	15722
query68	9270	781	532	532
query69	453	307	267	267
query70	1217	1178	1112	1112
query71	422	291	258	258
query72	5945	3867	3925	3867
query73	705	790	370	370
query74	9954	9377	8774	8774
query75	4571	3167	2689	2689
query76	5160	1235	859	859
query77	980	396	272	272
query78	10000	10636	9419	9419
query79	4548	885	603	603
query80	730	541	436	436
query81	476	265	235	235
query82	312	149	131	131
query83	193	170	150	150
query84	287	96	74	74
query85	782	364	312	312
query86	367	323	272	272
query87	4614	4643	4339	4339
query88	3621	2262	2224	2224
query89	418	338	300	300
query90	2012	191	189	189
query91	141	133	108	108
query92	70	64	53	53
query93	2534	911	544	544
query94	675	399	289	289
query95	340	257	262	257
query96	486	626	283	283
query97	2836	2831	2694	2694
query98	220	199	205	199
query99	1653	1561	1448	1448
Total cold run time: 298199 ms
Total hot run time: 190849 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.9 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ef56db55fbb6b553428b0d9fdefdbd5df08d3e60, data reload: false

query1	0.04	0.06	0.03
query2	0.07	0.03	0.03
query3	0.23	0.08	0.08
query4	1.61	0.10	0.10
query5	0.43	0.41	0.42
query6	1.18	0.64	0.65
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.58	0.54	0.51
query10	0.57	0.59	0.56
query11	0.16	0.10	0.10
query12	0.15	0.12	0.12
query13	0.61	0.62	0.61
query14	2.71	2.74	2.76
query15	0.89	0.82	0.83
query16	0.40	0.38	0.38
query17	1.06	1.07	1.05
query18	0.22	0.22	0.21
query19	1.91	1.97	1.81
query20	0.01	0.01	0.02
query21	15.36	0.96	0.60
query22	0.74	0.80	0.79
query23	15.13	1.42	0.53
query24	2.55	1.46	1.26
query25	0.19	0.16	0.27
query26	0.23	0.14	0.14
query27	0.08	0.05	0.06
query28	13.90	1.52	1.04
query29	12.57	4.04	3.30
query30	0.25	0.09	0.06
query31	2.84	0.61	0.39
query32	3.23	0.55	0.46
query33	3.08	3.09	3.10
query34	16.63	5.10	4.52
query35	4.49	4.45	4.47
query36	0.64	0.49	0.48
query37	0.09	0.07	0.06
query38	0.04	0.04	0.04
query39	0.04	0.02	0.02
query40	0.17	0.13	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 105.29 s
Total hot run time: 31.9 s

@zy-kkk
Copy link
Member Author

zy-kkk commented Dec 27, 2024

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Dec 27, 2024
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 27, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 32453 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ce7b8e6f974b01a9b694662a1b85c95e0b358164, data reload: false

------ Round 1 ----------------------------------
q1	17579	6765	6011	6011
q2	2046	299	176	176
q3	10415	1245	705	705
q4	10214	872	437	437
q5	7517	2192	1953	1953
q6	204	184	153	153
q7	885	751	619	619
q8	9260	1369	1154	1154
q9	5199	4885	4888	4885
q10	6750	2311	1855	1855
q11	484	273	261	261
q12	346	368	217	217
q13	17763	3634	2945	2945
q14	219	235	218	218
q15	568	516	498	498
q16	647	614	585	585
q17	544	840	311	311
q18	7083	6593	6385	6385
q19	1723	964	582	582
q20	323	323	199	199
q21	2802	2149	1993	1993
q22	361	330	311	311
Total cold run time: 102932 ms
Total hot run time: 32453 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6306	6223	6229	6223
q2	236	322	229	229
q3	2225	2665	2371	2371
q4	1431	1809	1398	1398
q5	4332	4722	4792	4722
q6	184	183	151	151
q7	2133	1966	1819	1819
q8	2588	2807	2666	2666
q9	7328	7252	7163	7163
q10	3053	3337	2813	2813
q11	598	507	497	497
q12	648	762	660	660
q13	3441	3686	3105	3105
q14	282	307	306	306
q15	573	514	506	506
q16	645	681	633	633
q17	1191	1722	1254	1254
q18	7651	7414	7126	7126
q19	789	1119	1031	1031
q20	1892	1964	1843	1843
q21	5441	5076	4868	4868
q22	599	633	613	613
Total cold run time: 53566 ms
Total hot run time: 51997 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190167 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ce7b8e6f974b01a9b694662a1b85c95e0b358164, data reload: false

query1	967	390	381	381
query2	6524	2435	2312	2312
query3	6702	210	210	210
query4	33798	23788	23664	23664
query5	4286	622	464	464
query6	305	205	197	197
query7	4627	492	296	296
query8	312	245	232	232
query9	9575	2760	2763	2760
query10	470	304	248	248
query11	18695	15261	15390	15261
query12	161	106	108	106
query13	1658	548	415	415
query14	11211	6817	7252	6817
query15	247	202	192	192
query16	8011	604	474	474
query17	1574	729	550	550
query18	2081	400	288	288
query19	218	176	149	149
query20	124	111	113	111
query21	214	118	112	112
query22	4403	4321	4617	4321
query23	34046	33278	33255	33255
query24	6362	2258	2295	2258
query25	449	433	371	371
query26	1208	261	150	150
query27	2028	467	330	330
query28	5291	2476	2449	2449
query29	671	533	436	436
query30	220	180	150	150
query31	974	902	831	831
query32	74	57	57	57
query33	503	346	288	288
query34	740	837	499	499
query35	800	812	762	762
query36	1000	1047	956	956
query37	123	96	71	71
query38	4308	4249	4122	4122
query39	1508	1454	1430	1430
query40	204	109	106	106
query41	48	46	46	46
query42	121	97	100	97
query43	525	522	487	487
query44	1322	795	800	795
query45	181	176	161	161
query46	855	1030	640	640
query47	1927	1950	1880	1880
query48	376	389	337	337
query49	769	465	381	381
query50	609	628	380	380
query51	7251	7065	6957	6957
query52	99	102	88	88
query53	222	253	179	179
query54	469	460	404	404
query55	89	80	79	79
query56	265	249	248	248
query57	1180	1196	1109	1109
query58	244	226	224	224
query59	3226	3195	3092	3092
query60	275	254	253	253
query61	118	104	109	104
query62	881	817	749	749
query63	220	191	207	191
query64	4185	995	715	715
query65	3315	3189	3237	3189
query66	1050	421	338	338
query67	15815	15963	15543	15543
query68	9181	743	497	497
query69	464	286	252	252
query70	1284	1162	1158	1158
query71	440	293	255	255
query72	6013	3785	3863	3785
query73	673	751	353	353
query74	9460	8964	9119	8964
query75	4457	3154	2639	2639
query76	5417	1192	773	773
query77	994	361	270	270
query78	10165	10134	9348	9348
query79	3243	902	587	587
query80	713	514	431	431
query81	470	269	223	223
query82	638	208	124	124
query83	193	167	144	144
query84	294	90	74	74
query85	784	369	304	304
query86	354	318	315	315
query87	4566	4560	4417	4417
query88	3387	2238	2191	2191
query89	420	348	300	300
query90	2019	189	188	188
query91	137	133	102	102
query92	66	58	53	53
query93	1820	856	512	512
query94	649	398	296	296
query95	327	258	248	248
query96	478	602	276	276
query97	2795	2772	2711	2711
query98	216	203	199	199
query99	1653	1579	1431	1431
Total cold run time: 296079 ms
Total hot run time: 190167 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.81 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ce7b8e6f974b01a9b694662a1b85c95e0b358164, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.03
query3	0.24	0.08	0.07
query4	1.60	0.10	0.10
query5	0.41	0.40	0.43
query6	1.16	0.68	0.65
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.57	0.51	0.51
query10	0.56	0.57	0.55
query11	0.15	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.59
query14	2.72	2.77	2.85
query15	0.90	0.82	0.82
query16	0.39	0.38	0.39
query17	0.96	1.01	1.03
query18	0.22	0.20	0.21
query19	1.98	1.84	1.94
query20	0.01	0.01	0.01
query21	15.36	0.88	0.57
query22	0.78	0.81	0.71
query23	15.15	1.48	0.57
query24	3.02	0.43	1.98
query25	0.23	0.16	0.16
query26	0.20	0.15	0.14
query27	0.07	0.07	0.04
query28	13.98	1.53	1.05
query29	12.59	3.95	3.26
query30	0.25	0.09	0.06
query31	2.82	0.60	0.37
query32	3.23	0.57	0.47
query33	3.13	3.06	3.21
query34	16.66	5.12	4.47
query35	4.52	4.46	4.45
query36	0.67	0.49	0.49
query37	0.09	0.06	0.05
query38	0.04	0.04	0.03
query39	0.04	0.02	0.03
query40	0.16	0.13	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 105.93 s
Total hot run time: 30.81 s

morningman
morningman approved these changes Dec 27, 2024
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morningman morningman merged commit 2b5c9c0 into apache:master Dec 27, 2024
26 of 28 checks passed
github-actions bot pushed a commit that referenced this pull request Dec 27, 2024
@zy-kkk
[fix](jdbc catalog) Get statistics using remote name (#46040)
c836102 
### What problem does this PR solve?

Related PR: #41510

Problem Summary:

We should use remote name to get statistics
@github-actions github-actions bot mentioned this pull request Dec 27, 2024
Merged
@zy-kkk zy-kkk deleted the fix_jdbc_stats branch December 27, 2024 10:06
zy-kkk added a commit to zy-kkk/doris that referenced this pull request Dec 27, 2024
@zy-kkk
branch-2.1: [fix](jdbc catalog) Get statistics using remote name apac…
60c91a9 
…he#46040
@zy-kkk zy-kkk mentioned this pull request Dec 27, 2024
Merged
morningman pushed a commit that referenced this pull request Dec 27, 2024
@zy-kkk
branch-2.1: [fix](jdbc catalog) Get statistics using remote name #46040…
75e9aa3 
… (#46116)

Cherry-picked from #46040
morningman pushed a commit that referenced this pull request Dec 27, 2024
@github-actions @zy-kkk
branch-3.0: [fix](jdbc catalog) Get statistics using remote name #46040…
865d30d 
… (#46113)

Cherry-picked from #46040

Co-authored-by: zy-kkk <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Jibing-Li Jibing-Li Jibing-Li approved these changes

@morningman morningman morningman approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.4-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@zy-kkk @Thearas @doris-robot @morningman @Jibing-Li @wm1581066