Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

branch-3.0: [opt](nereids) support pushing down aggr distinct through join #43380 #45067

Merged
merged 1 commit into from
Dec 10, 2024

Conversation

github-actions[bot]
Copy link
Contributor

@github-actions github-actions bot commented Dec 5, 2024

Cherry-picked from #43380

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Dec 5, 2024
@dataroaring dataroaring reopened this Dec 5, 2024
@doris-robot
Copy link

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40598 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 986af6488a75a63fb0745fd094ed35a1500b5f9d, data reload: false

------ Round 1 ----------------------------------
q1	17567	7463	7373	7373
q2	2063	166	158	158
q3	10843	1113	1167	1113
q4	10554	754	784	754
q5	7726	2798	2753	2753
q6	233	148	150	148
q7	963	608	607	607
q8	9338	1928	2002	1928
q9	6872	6405	6432	6405
q10	7022	2308	2318	2308
q11	453	260	267	260
q12	404	219	216	216
q13	17791	2980	2961	2961
q14	243	219	217	217
q15	563	520	516	516
q16	667	616	597	597
q17	952	527	570	527
q18	7260	6547	6513	6513
q19	3930	1052	982	982
q20	454	188	185	185
q21	3936	3060	3111	3060
q22	1077	1026	1017	1017
Total cold run time: 110911 ms
Total hot run time: 40598 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7429	7368	7293	7293
q2	327	228	221	221
q3	2973	2887	2865	2865
q4	2035	1728	1792	1728
q5	5685	5695	5764	5695
q6	218	145	138	138
q7	2164	1748	1739	1739
q8	3384	3519	3409	3409
q9	8881	8872	8850	8850
q10	3561	3513	3509	3509
q11	596	504	498	498
q12	804	604	589	589
q13	16461	3112	3155	3112
q14	296	270	263	263
q15	573	523	520	520
q16	694	664	668	664
q17	1835	1634	1597	1597
q18	8209	7736	7409	7409
q19	5576	1681	1611	1611
q20	2066	1852	1823	1823
q21	5340	5364	5338	5338
q22	1097	992	1019	992
Total cold run time: 80204 ms
Total hot run time: 59863 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 195215 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 986af6488a75a63fb0745fd094ed35a1500b5f9d, data reload: false

query1	1231	924	924	924
query2	6246	2087	2024	2024
query3	10809	3937	4002	3937
query4	65796	28927	23665	23665
query5	5272	479	440	440
query6	431	180	179	179
query7	5713	315	319	315
query8	319	236	234	234
query9	9432	2705	2653	2653
query10	512	267	253	253
query11	17872	15146	15735	15146
query12	176	103	102	102
query13	1566	433	416	416
query14	10891	6424	7422	6424
query15	212	169	171	169
query16	7252	498	513	498
query17	1038	573	621	573
query18	1895	337	316	316
query19	239	156	165	156
query20	119	110	109	109
query21	203	108	103	103
query22	4568	4338	4407	4338
query23	34853	34374	34227	34227
query24	5997	2932	2928	2928
query25	548	423	409	409
query26	685	169	180	169
query27	1785	302	309	302
query28	4529	2617	2518	2518
query29	706	473	459	459
query30	244	163	161	161
query31	1008	879	836	836
query32	68	55	58	55
query33	437	310	302	302
query34	926	506	537	506
query35	843	751	729	729
query36	1082	950	980	950
query37	117	73	70	70
query38	4102	3886	4021	3886
query39	1561	1460	1488	1460
query40	207	102	103	102
query41	51	50	51	50
query42	111	104	99	99
query43	546	495	490	490
query44	1152	813	815	813
query45	182	169	171	169
query46	1156	728	733	728
query47	1991	1897	1866	1866
query48	486	375	373	373
query49	709	362	377	362
query50	831	417	397	397
query51	7323	7173	7270	7173
query52	94	96	89	89
query53	261	180	189	180
query54	541	455	441	441
query55	74	77	74	74
query56	252	222	225	222
query57	1169	1092	1103	1092
query58	203	194	204	194
query59	3065	2853	2980	2853
query60	296	252	251	251
query61	106	107	109	107
query62	765	669	674	669
query63	205	189	195	189
query64	1793	658	650	650
query65	3233	3192	3147	3147
query66	644	290	315	290
query67	15812	15357	15332	15332
query68	4683	555	556	555
query69	426	247	252	247
query70	1161	1132	1144	1132
query71	375	255	248	248
query72	6503	3720	3895	3720
query73	760	344	333	333
query74	10056	8877	8909	8877
query75	3331	2645	2612	2612
query76	2224	1128	969	969
query77	506	260	272	260
query78	10641	9636	9573	9573
query79	8068	592	578	578
query80	2319	422	429	422
query81	552	246	237	237
query82	1346	116	113	113
query83	254	141	142	141
query84	290	80	75	75
query85	1810	306	304	304
query86	498	302	267	267
query87	4474	4260	4211	4211
query88	5676	2385	2377	2377
query89	410	291	293	291
query90	2078	190	187	187
query91	182	175	140	140
query92	67	49	48	48
query93	6531	538	533	533
query94	1038	291	277	277
query95	345	252	250	250
query96	663	275	280	275
query97	3416	3197	3115	3115
query98	226	200	207	200
query99	1648	1316	1305	1305
Total cold run time: 337098 ms
Total hot run time: 195215 ms

@xzj7019
Copy link
Contributor

xzj7019 commented Dec 10, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40754 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 986af6488a75a63fb0745fd094ed35a1500b5f9d, data reload: false

------ Round 1 ----------------------------------
q1	18011	7723	7415	7415
q2	2761	163	176	163
q3	11679	1164	1200	1164
q4	11440	792	753	753
q5	7750	2856	2844	2844
q6	241	153	153	153
q7	1007	623	618	618
q8	9744	1930	2042	1930
q9	6842	6414	6399	6399
q10	7066	2286	2307	2286
q11	463	255	258	255
q12	403	215	209	209
q13	17793	2965	2985	2965
q14	230	205	204	204
q15	555	522	505	505
q16	675	621	612	612
q17	982	585	555	555
q18	7301	6512	6459	6459
q19	1831	1028	1037	1028
q20	470	200	215	200
q21	3892	3142	3078	3078
q22	1061	959	983	959
Total cold run time: 112197 ms
Total hot run time: 40754 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7362	7234	7281	7234
q2	349	246	235	235
q3	3137	2952	2851	2851
q4	2002	1794	1806	1794
q5	5671	5721	5691	5691
q6	231	143	145	143
q7	2189	1728	1742	1728
q8	3319	3470	3503	3470
q9	8947	8809	8835	8809
q10	3621	3498	3510	3498
q11	596	512	521	512
q12	789	578	583	578
q13	16953	3003	2974	2974
q14	280	270	255	255
q15	545	515	501	501
q16	697	673	659	659
q17	1798	1566	1539	1539
q18	7680	7409	7231	7231
q19	5439	1538	1523	1523
q20	2001	1746	1786	1746
q21	5121	4991	4897	4897
q22	1057	949	986	949
Total cold run time: 79784 ms
Total hot run time: 58817 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189923 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 986af6488a75a63fb0745fd094ed35a1500b5f9d, data reload: false

query1	979	375	364	364
query2	6507	2070	2064	2064
query3	6703	229	224	224
query4	34030	23453	23866	23453
query5	4351	441	434	434
query6	266	180	174	174
query7	4623	311	313	311
query8	312	229	227	227
query9	9427	2658	2641	2641
query10	473	271	260	260
query11	18373	15206	15170	15170
query12	150	99	102	99
query13	1634	422	418	418
query14	8697	7083	6438	6438
query15	213	170	169	169
query16	7975	488	489	488
query17	1639	576	554	554
query18	2071	312	313	312
query19	203	163	153	153
query20	116	108	107	107
query21	68	48	53	48
query22	4583	4402	4431	4402
query23	34451	34112	33732	33732
query24	11756	2816	2837	2816
query25	712	406	400	400
query26	1846	166	165	165
query27	3016	296	287	287
query28	8340	2457	2437	2437
query29	1105	466	439	439
query30	330	165	165	165
query31	1023	806	796	796
query32	99	58	59	58
query33	800	291	296	291
query34	1012	485	501	485
query35	881	732	719	719
query36	1091	940	928	928
query37	286	76	73	73
query38	4169	3861	3879	3861
query39	1504	1438	1421	1421
query40	228	85	91	85
query41	61	49	49	49
query42	118	102	100	100
query43	516	482	487	482
query44	1281	785	784	784
query45	185	169	179	169
query46	1137	722	731	722
query47	1887	1813	1826	1813
query48	461	392	404	392
query49	1314	390	385	385
query50	832	404	434	404
query51	7366	7069	7052	7052
query52	102	89	92	89
query53	266	189	183	183
query54	1285	443	446	443
query55	80	74	75	74
query56	260	236	225	225
query57	1199	1103	1064	1064
query58	235	200	203	200
query59	3142	2938	2788	2788
query60	282	253	246	246
query61	109	107	108	107
query62	863	681	672	672
query63	214	187	186	186
query64	5291	663	636	636
query65	3305	3209	3181	3181
query66	1439	312	307	307
query67	15752	15510	15390	15390
query68	4917	556	562	556
query69	429	269	262	262
query70	1151	1071	1143	1071
query71	414	250	244	244
query72	6433	3999	3970	3970
query73	775	340	341	340
query74	10451	8849	8866	8849
query75	3389	2614	2616	2614
query76	2907	1061	1094	1061
query77	418	269	273	269
query78	10598	9790	9505	9505
query79	7087	597	598	597
query80	1697	437	419	419
query81	577	240	234	234
query82	978	124	115	115
query83	266	152	147	147
query84	305	87	90	87
query85	2236	299	285	285
query86	494	297	280	280
query87	4440	4253	4190	4190
query88	5469	2389	2395	2389
query89	434	289	298	289
query90	2169	184	190	184
query91	181	143	146	143
query92	60	48	49	48
query93	5363	562	534	534
query94	1000	286	289	286
query95	349	259	247	247
query96	630	291	279	279
query97	3315	3161	3129	3129
query98	221	208	190	190
query99	1629	1299	1334	1299
Total cold run time: 315806 ms
Total hot run time: 189923 ms

@morrySnow morrySnow merged commit 243561d into branch-3.0 Dec 10, 2024
19 of 21 checks passed
@morrySnow morrySnow deleted the auto-pick-43380-branch-3.0 branch December 10, 2024 07:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants