Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](nereids) not rewrite first_value when first_value ignore nulls #45065

Merged
merged 5 commits into from
Dec 10, 2024

Conversation

feiniaofeiafei
Copy link
Contributor

@feiniaofeiafei feiniaofeiafei commented Dec 5, 2024

What problem does this PR solve?

this pr fix 2 things:
1.when first_value/last_value second parameter is true, which means ignore nulls, fe should not do some rewrite.
2.adding check that first_value/last_value second parameter must be true or false.

Issue Number: close #xxx

Related PR: #44996 #27623

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@feiniaofeiafei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39776 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8c163a1f9de9c55467ce06b1b288200f1eec0c26, data reload: false

------ Round 1 ----------------------------------
q1	17602	7643	7288	7288
q2	2076	173	167	167
q3	10710	1138	1132	1132
q4	10466	779	729	729
q5	7599	2675	2753	2675
q6	239	147	148	147
q7	998	629	610	610
q8	9254	1833	1898	1833
q9	6631	6482	6473	6473
q10	6983	2316	2289	2289
q11	452	266	253	253
q12	407	219	212	212
q13	17768	3008	3017	3008
q14	240	211	206	206
q15	586	527	536	527
q16	651	591	599	591
q17	973	513	502	502
q18	7318	6619	6685	6619
q19	1354	1053	904	904
q20	476	181	188	181
q21	3963	3128	3125	3125
q22	380	305	325	305
Total cold run time: 107126 ms
Total hot run time: 39776 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7266	7189	7196	7189
q2	324	236	236	236
q3	2835	2786	3028	2786
q4	2131	1827	1776	1776
q5	5538	5712	5662	5662
q6	222	141	140	140
q7	2224	1809	1785	1785
q8	3330	3595	3463	3463
q9	9010	8985	9011	8985
q10	3604	3538	3546	3538
q11	580	499	504	499
q12	809	612	612	612
q13	12187	3265	3193	3193
q14	290	265	282	265
q15	578	517	518	517
q16	684	654	621	621
q17	1780	1599	1576	1576
q18	7780	7301	7264	7264
q19	1645	1552	1423	1423
q20	2050	1839	1845	1839
q21	5367	5328	5143	5143
q22	604	587	533	533
Total cold run time: 70838 ms
Total hot run time: 59045 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190850 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8c163a1f9de9c55467ce06b1b288200f1eec0c26, data reload: false

query1	1153	373	375	373
query2	6541	2109	1944	1944
query3	6708	218	217	217
query4	34173	23861	23465	23465
query5	4350	476	482	476
query6	287	190	189	189
query7	4627	308	314	308
query8	297	249	237	237
query9	9724	2722	2721	2721
query10	481	250	250	250
query11	18137	15349	15250	15250
query12	146	99	107	99
query13	1651	432	404	404
query14	9646	7397	6621	6621
query15	258	190	178	178
query16	8181	465	446	446
query17	1652	575	576	575
query18	2144	304	304	304
query19	371	156	156	156
query20	120	115	110	110
query21	213	104	111	104
query22	4546	4339	4348	4339
query23	34923	34187	34542	34187
query24	11426	2527	2441	2441
query25	710	406	409	406
query26	1815	158	161	158
query27	2777	287	293	287
query28	8008	2474	2449	2449
query29	1026	414	406	406
query30	301	151	158	151
query31	1057	854	856	854
query32	94	61	61	61
query33	781	319	286	286
query34	957	509	543	509
query35	900	750	752	750
query36	1085	930	948	930
query37	182	77	73	73
query38	4398	4201	4226	4201
query39	1495	1444	1409	1409
query40	283	103	98	98
query41	47	43	45	43
query42	108	100	99	99
query43	575	471	487	471
query44	1230	808	802	802
query45	186	168	161	161
query46	1157	753	735	735
query47	1957	1839	1821	1821
query48	408	312	316	312
query49	1271	397	393	393
query50	817	393	388	388
query51	7180	7016	7137	7016
query52	106	100	96	96
query53	259	188	182	182
query54	1158	412	409	409
query55	94	76	79	76
query56	275	230	243	230
query57	1247	1099	1095	1095
query58	236	213	216	213
query59	3369	3044	2899	2899
query60	281	260	270	260
query61	122	102	109	102
query62	871	677	671	671
query63	219	209	199	199
query64	5202	663	689	663
query65	3303	3234	3285	3234
query66	1450	322	308	308
query67	15942	15501	15423	15423
query68	4904	553	571	553
query69	429	254	260	254
query70	1218	1170	1157	1157
query71	339	269	251	251
query72	6392	4061	4056	4056
query73	770	358	371	358
query74	9407	8974	9047	8974
query75	3483	2685	2688	2685
query76	2887	1085	1113	1085
query77	422	279	278	278
query78	10396	9502	9387	9387
query79	2479	619	617	617
query80	1162	423	431	423
query81	534	235	233	233
query82	666	118	120	118
query83	255	149	154	149
query84	241	71	71	71
query85	1734	301	302	301
query86	484	309	304	304
query87	4639	4504	4593	4504
query88	4159	2219	2194	2194
query89	398	298	302	298
query90	2009	190	195	190
query91	138	105	103	103
query92	66	54	54	54
query93	1598	555	538	538
query94	972	297	293	293
query95	362	258	255	255
query96	619	280	280	280
query97	2871	2678	2679	2678
query98	219	195	200	195
query99	1537	1332	1291	1291
Total cold run time: 304483 ms
Total hot run time: 190850 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.96 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8c163a1f9de9c55467ce06b1b288200f1eec0c26, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.61	0.10	0.11
query5	0.43	0.40	0.41
query6	1.16	0.67	0.65
query7	0.02	0.02	0.01
query8	0.04	0.04	0.03
query9	0.58	0.50	0.51
query10	0.57	0.56	0.55
query11	0.14	0.10	0.11
query12	0.14	0.11	0.11
query13	0.61	0.61	0.60
query14	2.77	2.87	2.85
query15	0.92	0.83	0.82
query16	0.37	0.39	0.38
query17	1.05	1.05	1.03
query18	0.22	0.22	0.21
query19	1.93	1.75	1.94
query20	0.02	0.01	0.01
query21	15.38	0.59	0.58
query22	2.45	1.76	1.69
query23	16.95	0.88	0.86
query24	2.66	1.63	0.41
query25	0.21	0.08	0.05
query26	0.41	0.14	0.14
query27	0.04	0.04	0.05
query28	10.89	1.10	1.07
query29	12.62	3.22	3.21
query30	0.25	0.07	0.07
query31	2.85	0.39	0.38
query32	3.26	0.47	0.46
query33	3.01	3.01	3.10
query34	17.01	4.48	4.46
query35	4.56	4.52	4.51
query36	0.66	0.48	0.52
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.18	0.13	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.02
Total cold run time: 106.63 s
Total hot run time: 31.96 s

@feiniaofeiafei
Copy link
Contributor Author

run cloud_p0

1 similar comment
@feiniaofeiafei
Copy link
Contributor Author

run cloud_p0

@feiniaofeiafei feiniaofeiafei force-pushed the fix_window_fe branch 2 times, most recently from 9e92b94 to 1b7d53f Compare December 9, 2024 07:02
@feiniaofeiafei
Copy link
Contributor Author

run buildall

@feiniaofeiafei feiniaofeiafei force-pushed the fix_window_fe branch 2 times, most recently from 33fe9d3 to 457d16d Compare December 9, 2024 07:24
@feiniaofeiafei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39933 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 457d16dbb708cfd7662d96c0dee3675f554e78c3, data reload: false

------ Round 1 ----------------------------------
q1	17644	8380	7270	7270
q2	2058	180	189	180
q3	10572	1061	1157	1061
q4	10582	773	730	730
q5	7589	2729	2739	2729
q6	242	154	149	149
q7	1016	625	601	601
q8	9243	1863	1923	1863
q9	6716	6539	6467	6467
q10	7026	2319	2338	2319
q11	479	265	257	257
q12	424	223	224	223
q13	17782	3012	2975	2975
q14	247	213	208	208
q15	586	520	524	520
q16	669	583	605	583
q17	994	592	544	544
q18	7317	6746	6670	6670
q19	1353	1060	985	985
q20	470	177	174	174
q21	4091	3376	3108	3108
q22	378	317	321	317
Total cold run time: 107478 ms
Total hot run time: 39933 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7297	7271	7224	7224
q2	324	223	227	223
q3	2946	2923	2930	2923
q4	2085	1839	1814	1814
q5	5618	5649	5680	5649
q6	223	134	144	134
q7	2266	1808	1795	1795
q8	3396	3531	3526	3526
q9	8907	9047	9029	9029
q10	3586	3565	3575	3565
q11	602	510	505	505
q12	831	629	616	616
q13	12406	3231	3221	3221
q14	303	271	274	271
q15	591	517	525	517
q16	709	655	651	651
q17	1856	1636	1609	1609
q18	8315	7848	7735	7735
q19	1705	1497	1589	1497
q20	2101	1904	1876	1876
q21	5579	5511	5457	5457
q22	665	585	547	547
Total cold run time: 72311 ms
Total hot run time: 60384 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197176 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 457d16dbb708cfd7662d96c0dee3675f554e78c3, data reload: false

query1	1253	964	926	926
query2	6246	2016	1949	1949
query3	10934	4377	4335	4335
query4	66893	29027	23578	23578
query5	5020	458	457	457
query6	438	187	179	179
query7	5694	305	292	292
query8	331	243	240	240
query9	9383	2722	2726	2722
query10	463	245	234	234
query11	17692	15339	16082	15339
query12	158	100	101	100
query13	1582	406	401	401
query14	10655	7023	7233	7023
query15	218	185	214	185
query16	7571	478	506	478
query17	1084	589	596	589
query18	1865	308	312	308
query19	237	149	161	149
query20	117	118	118	118
query21	217	103	108	103
query22	4996	4457	4618	4457
query23	34921	34681	35278	34681
query24	5498	2508	2431	2431
query25	490	386	378	378
query26	658	155	150	150
query27	1924	292	284	284
query28	4446	2526	2471	2471
query29	676	430	404	404
query30	212	151	152	151
query31	1020	850	859	850
query32	66	58	56	56
query33	440	291	289	289
query34	944	508	514	508
query35	898	772	767	767
query36	1096	963	973	963
query37	122	74	96	74
query38	4403	4364	4411	4364
query39	1510	1482	1476	1476
query40	211	96	95	95
query41	45	42	42	42
query42	103	102	98	98
query43	539	507	494	494
query44	1216	823	848	823
query45	190	166	177	166
query46	1143	734	761	734
query47	2033	1963	1900	1900
query48	428	323	333	323
query49	717	396	402	396
query50	840	401	380	380
query51	7276	7259	7036	7036
query52	98	89	83	83
query53	251	175	175	175
query54	506	398	392	392
query55	80	77	79	77
query56	251	228	232	228
query57	1239	1145	1122	1122
query58	215	204	210	204
query59	3274	2929	3199	2929
query60	274	234	257	234
query61	129	106	104	104
query62	792	679	679	679
query63	212	180	194	180
query64	1368	663	628	628
query65	3286	3203	3205	3203
query66	633	299	300	299
query67	15993	15654	15768	15654
query68	4162	582	568	568
query69	399	252	247	247
query70	1136	1166	1134	1134
query71	328	259	249	249
query72	6504	4136	4111	4111
query73	759	363	367	363
query74	10070	8957	8947	8947
query75	3373	2651	2704	2651
query76	1852	1100	1236	1100
query77	497	273	266	266
query78	10560	9472	9580	9472
query79	1541	595	596	595
query80	878	421	429	421
query81	495	228	227	227
query82	1275	113	116	113
query83	228	143	139	139
query84	277	75	69	69
query85	897	305	297	297
query86	350	311	280	280
query87	4869	4578	4499	4499
query88	3531	2218	2183	2183
query89	420	296	282	282
query90	1912	182	184	182
query91	139	131	99	99
query92	68	47	50	47
query93	1999	542	540	540
query94	739	280	294	280
query95	343	246	244	244
query96	630	274	286	274
query97	2872	2656	2672	2656
query98	225	199	187	187
query99	1624	1317	1332	1317
Total cold run time: 320665 ms
Total hot run time: 197176 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.95 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 457d16dbb708cfd7662d96c0dee3675f554e78c3, data reload: false

query1	0.04	0.03	0.02
query2	0.07	0.03	0.03
query3	0.23	0.08	0.06
query4	1.64	0.10	0.10
query5	0.42	0.42	0.43
query6	1.15	0.67	0.66
query7	0.02	0.02	0.02
query8	0.05	0.03	0.03
query9	0.58	0.50	0.52
query10	0.56	0.56	0.54
query11	0.14	0.11	0.10
query12	0.14	0.12	0.12
query13	0.62	0.60	0.60
query14	2.74	2.75	2.83
query15	0.90	0.82	0.82
query16	0.38	0.36	0.38
query17	1.00	0.99	1.01
query18	0.23	0.22	0.21
query19	1.98	1.82	1.99
query20	0.01	0.01	0.01
query21	15.35	0.60	0.58
query22	2.86	2.24	1.80
query23	16.98	1.18	0.83
query24	3.16	0.45	1.00
query25	0.33	0.24	0.04
query26	0.25	0.15	0.14
query27	0.04	0.05	0.04
query28	11.17	1.11	1.07
query29	12.54	3.25	3.26
query30	0.24	0.07	0.06
query31	2.88	0.38	0.37
query32	3.28	0.45	0.47
query33	3.08	3.01	3.06
query34	16.80	4.49	4.44
query35	4.47	4.51	4.47
query36	0.69	0.48	0.48
query37	0.09	0.06	0.07
query38	0.05	0.03	0.03
query39	0.03	0.02	0.02
query40	0.17	0.13	0.13
query41	0.09	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 107.51 s
Total hot run time: 31.95 s

@feiniaofeiafei
Copy link
Contributor Author

run buildall

Comment on lines 320 to 322
if (!firstValue.child(1).isConstant()) {
throw new AnalysisException("The second parameter of first_value must be constant");
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we should add this check in org.apache.doris.nereids.trees.expressions.functions.window.FirstOrLastValue.
let impl checkLegalityBeforeTypeCoercion

@doris-robot
Copy link

TPC-H: Total hot run time: 39821 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 286d2c8887b1c87eb90514d90973127662fa9b72, data reload: false

------ Round 1 ----------------------------------
q1	17586	7427	7300	7300
q2	2041	180	175	175
q3	10586	1103	1155	1103
q4	10571	719	767	719
q5	7641	2724	2726	2724
q6	239	149	151	149
q7	994	639	604	604
q8	9241	1884	1944	1884
q9	6686	6386	6431	6386
q10	7036	2299	2347	2299
q11	485	258	257	257
q12	432	220	223	220
q13	17779	3010	3059	3010
q14	254	214	218	214
q15	579	521	522	521
q16	647	589	573	573
q17	991	555	483	483
q18	7215	6665	6667	6665
q19	1334	1036	966	966
q20	471	181	185	181
q21	4041	3255	3065	3065
q22	383	329	323	323
Total cold run time: 107232 ms
Total hot run time: 39821 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7225	7243	7236	7236
q2	332	225	234	225
q3	2940	2761	2953	2761
q4	2070	1877	1830	1830
q5	5619	5758	5664	5664
q6	224	147	144	144
q7	2269	1815	1782	1782
q8	3347	3549	3513	3513
q9	9085	9073	9032	9032
q10	3610	3538	3532	3532
q11	595	515	508	508
q12	837	612	589	589
q13	13139	3247	3231	3231
q14	307	274	276	274
q15	588	535	531	531
q16	676	651	630	630
q17	1895	1621	1589	1589
q18	8350	7762	7710	7710
q19	1774	1520	1608	1520
q20	2106	1853	1856	1853
q21	5595	5382	5446	5382
q22	669	577	589	577
Total cold run time: 73252 ms
Total hot run time: 60113 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197688 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 286d2c8887b1c87eb90514d90973127662fa9b72, data reload: false

query1	1251	934	944	934
query2	6245	2050	2041	2041
query3	10964	4422	4630	4422
query4	33028	23494	23500	23494
query5	3555	488	475	475
query6	278	189	185	185
query7	3974	302	319	302
query8	307	244	233	233
query9	9696	2706	2702	2702
query10	470	259	259	259
query11	17968	15207	15181	15181
query12	158	101	102	101
query13	1546	421	394	394
query14	9691	7203	7241	7203
query15	285	205	188	188
query16	8048	457	475	457
query17	1605	579	605	579
query18	2183	300	319	300
query19	300	161	166	161
query20	118	116	118	116
query21	202	108	108	108
query22	4780	4603	4448	4448
query23	35266	34721	34908	34721
query24	10451	2542	2502	2502
query25	620	389	384	384
query26	1221	167	155	155
query27	2605	290	290	290
query28	7927	2478	2454	2454
query29	824	432	409	409
query30	237	153	160	153
query31	1061	852	829	829
query32	97	57	61	57
query33	786	310	311	310
query34	1093	534	540	534
query35	921	770	782	770
query36	1139	964	977	964
query37	146	80	80	80
query38	4566	4329	4418	4329
query39	1532	1465	1451	1451
query40	220	109	112	109
query41	47	48	48	48
query42	119	103	101	101
query43	581	491	483	483
query44	1275	855	850	850
query45	194	169	187	169
query46	1182	741	750	741
query47	2032	1910	1941	1910
query48	439	318	329	318
query49	956	394	380	380
query50	871	403	406	403
query51	7496	7261	7262	7261
query52	101	87	93	87
query53	266	183	187	183
query54	1335	440	407	407
query55	80	81	81	81
query56	261	249	252	249
query57	1341	1131	1162	1131
query58	228	219	215	215
query59	3248	3101	2987	2987
query60	265	238	232	232
query61	107	103	105	103
query62	884	669	660	660
query63	219	188	182	182
query64	4077	663	636	636
query65	3289	3178	3198	3178
query66	830	302	294	294
query67	16019	15607	15546	15546
query68	4850	544	557	544
query69	436	254	260	254
query70	1170	1136	1142	1136
query71	340	248	262	248
query72	6323	4097	4048	4048
query73	772	351	355	351
query74	10664	9144	9167	9144
query75	3424	2691	2672	2672
query76	2835	1026	1207	1026
query77	422	276	279	276
query78	10426	9479	9334	9334
query79	1216	600	614	600
query80	999	453	446	446
query81	514	240	232	232
query82	1305	124	112	112
query83	221	154	145	145
query84	238	69	69	69
query85	1316	303	291	291
query86	386	300	314	300
query87	4750	4533	4541	4533
query88	3060	2232	2179	2179
query89	422	298	291	291
query90	1941	188	182	182
query91	138	99	101	99
query92	66	50	49	49
query93	1190	534	535	534
query94	877	291	296	291
query95	349	252	248	248
query96	609	272	277	272
query97	2847	2673	2666	2666
query98	219	192	201	192
query99	1573	1305	1318	1305
Total cold run time: 301004 ms
Total hot run time: 197688 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.63 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 286d2c8887b1c87eb90514d90973127662fa9b72, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.61	0.11	0.10
query5	0.43	0.43	0.42
query6	1.17	0.67	0.65
query7	0.02	0.01	0.01
query8	0.04	0.03	0.04
query9	0.61	0.52	0.51
query10	0.55	0.54	0.57
query11	0.15	0.10	0.11
query12	0.15	0.12	0.10
query13	0.61	0.62	0.59
query14	2.82	2.76	2.86
query15	0.91	0.83	0.83
query16	0.38	0.40	0.40
query17	1.07	1.07	1.00
query18	0.24	0.21	0.21
query19	1.99	1.82	2.01
query20	0.02	0.01	0.01
query21	15.37	0.58	0.59
query22	3.07	1.73	1.22
query23	17.10	0.96	0.82
query24	2.88	1.57	1.68
query25	0.27	0.11	0.14
query26	0.50	0.14	0.13
query27	0.04	0.04	0.04
query28	9.69	1.11	1.07
query29	12.57	3.24	3.22
query30	0.24	0.06	0.06
query31	2.88	0.38	0.37
query32	3.26	0.47	0.47
query33	2.98	3.03	3.04
query34	16.96	4.46	4.48
query35	4.52	4.57	4.48
query36	0.65	0.49	0.48
query37	0.09	0.06	0.06
query38	0.04	0.04	0.04
query39	0.04	0.03	0.02
query40	0.16	0.12	0.12
query41	0.09	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.58 s
Total hot run time: 32.63 s

@feiniaofeiafei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40115 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 83969fd5ef08402c04abf771a116894f07920dc2, data reload: false

------ Round 1 ----------------------------------
q1	17573	7410	7301	7301
q2	2045	180	165	165
q3	10705	1080	1122	1080
q4	10216	779	821	779
q5	7585	2758	2720	2720
q6	239	148	145	145
q7	992	641	607	607
q8	9275	1839	1929	1839
q9	6610	6510	6532	6510
q10	7013	2293	2353	2293
q11	470	260	250	250
q12	432	224	222	222
q13	17755	3024	3055	3024
q14	250	215	207	207
q15	576	529	513	513
q16	656	596	583	583
q17	976	577	592	577
q18	7215	6632	6761	6632
q19	1338	1049	1067	1049
q20	446	182	175	175
q21	4080	3195	3125	3125
q22	368	331	319	319
Total cold run time: 106815 ms
Total hot run time: 40115 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7337	7337	7260	7260
q2	319	222	230	222
q3	2876	2779	3008	2779
q4	2045	1799	1836	1799
q5	5672	5676	5669	5669
q6	234	146	139	139
q7	2225	1805	1798	1798
q8	3393	3551	3477	3477
q9	8983	9075	9110	9075
q10	3644	3595	3553	3553
q11	609	506	510	506
q12	799	579	625	579
q13	11748	3297	3266	3266
q14	305	282	277	277
q15	592	514	526	514
q16	683	665	660	660
q17	1845	1605	1616	1605
q18	8263	7773	7705	7705
q19	1655	1610	1575	1575
q20	2123	1849	1877	1849
q21	5680	5590	5422	5422
q22	646	546	575	546
Total cold run time: 71676 ms
Total hot run time: 60275 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197242 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 83969fd5ef08402c04abf771a116894f07920dc2, data reload: false

query1	1276	942	928	928
query2	6228	2114	2045	2045
query3	11076	4439	4411	4411
query4	33083	23702	23513	23513
query5	3685	473	450	450
query6	312	185	200	185
query7	3986	304	310	304
query8	299	252	247	247
query9	9305	2719	2710	2710
query10	459	245	248	245
query11	17720	15245	15378	15245
query12	152	99	101	99
query13	1542	428	417	417
query14	9151	7577	6662	6662
query15	291	195	199	195
query16	7991	451	481	451
query17	1759	572	572	572
query18	2185	315	302	302
query19	363	148	144	144
query20	120	109	114	109
query21	210	113	109	109
query22	4893	4501	4458	4458
query23	35150	34534	34673	34534
query24	10327	2585	2628	2585
query25	589	391	390	390
query26	742	148	146	146
query27	2197	300	294	294
query28	6329	2495	2514	2495
query29	761	418	412	412
query30	234	152	148	148
query31	1035	868	872	868
query32	92	58	54	54
query33	796	310	304	304
query34	1042	503	549	503
query35	908	792	762	762
query36	1123	976	950	950
query37	140	80	79	79
query38	4606	4471	4526	4471
query39	1536	1465	1497	1465
query40	208	106	106	106
query41	48	51	47	47
query42	119	102	100	100
query43	597	502	514	502
query44	1294	826	827	826
query45	185	172	173	172
query46	1180	747	716	716
query47	2061	1937	1944	1937
query48	425	335	327	327
query49	922	398	408	398
query50	850	387	386	386
query51	7386	7364	7231	7231
query52	104	89	96	89
query53	258	200	217	200
query54	1230	410	410	410
query55	86	75	74	74
query56	258	243	236	236
query57	1253	1120	1124	1120
query58	221	206	207	206
query59	3198	2998	2986	2986
query60	281	239	249	239
query61	109	110	108	108
query62	860	676	689	676
query63	210	185	181	181
query64	3770	680	654	654
query65	3251	3159	3203	3159
query66	814	297	301	297
query67	16100	15586	15593	15586
query68	4548	569	559	559
query69	432	252	259	252
query70	1197	1151	1153	1151
query71	366	249	297	249
query72	6356	4065	3984	3984
query73	772	350	361	350
query74	10507	9137	8996	8996
query75	3417	2658	2678	2658
query76	2776	1013	1107	1013
query77	412	264	264	264
query78	10571	9657	9445	9445
query79	1591	601	599	599
query80	1190	423	439	423
query81	542	233	231	231
query82	969	124	120	120
query83	235	146	141	141
query84	222	66	69	66
query85	1301	299	293	293
query86	386	266	294	266
query87	4816	4671	4627	4627
query88	3368	2179	2169	2169
query89	426	291	300	291
query90	1917	186	186	186
query91	133	108	105	105
query92	57	49	50	49
query93	1881	546	536	536
query94	877	292	288	288
query95	351	248	249	248
query96	643	274	272	272
query97	2805	2666	2706	2666
query98	214	200	202	200
query99	1592	1315	1325	1315
Total cold run time: 297841 ms
Total hot run time: 197242 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.02 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 83969fd5ef08402c04abf771a116894f07920dc2, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.03	0.03
query3	0.23	0.08	0.07
query4	1.61	0.11	0.11
query5	0.41	0.41	0.44
query6	1.15	0.65	0.66
query7	0.02	0.01	0.02
query8	0.05	0.04	0.03
query9	0.58	0.52	0.50
query10	0.57	0.56	0.54
query11	0.15	0.11	0.10
query12	0.14	0.11	0.11
query13	0.62	0.61	0.60
query14	2.75	2.83	2.86
query15	0.91	0.83	0.83
query16	0.37	0.41	0.39
query17	1.05	1.03	1.08
query18	0.22	0.20	0.21
query19	1.95	1.81	2.04
query20	0.01	0.01	0.02
query21	15.37	0.58	0.58
query22	2.44	2.51	1.81
query23	16.95	1.16	0.81
query24	3.17	1.26	1.48
query25	0.18	0.09	0.07
query26	0.64	0.14	0.14
query27	0.05	0.05	0.04
query28	10.01	1.11	1.08
query29	12.54	3.23	3.23
query30	0.25	0.07	0.06
query31	2.86	0.39	0.37
query32	3.26	0.46	0.46
query33	2.97	3.00	3.01
query34	17.20	4.53	4.57
query35	4.54	4.51	4.55
query36	0.66	0.48	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.03	0.02
query40	0.16	0.13	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.47 s
Total hot run time: 33.02 s

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Dec 10, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 10, 2024
@morrySnow morrySnow merged commit 8ad6b28 into apache:master Dec 10, 2024
28 of 29 checks passed
github-actions bot pushed a commit that referenced this pull request Dec 10, 2024
…45065)

this pr fix 2 things:
1.when first_value/last_value second parameter is true, which means
ignore nulls, fe should not do some rewrite.
2.adding check that first_value/last_value second parameter must be true
or false.

Related PR: #44996 #27623
github-actions bot pushed a commit that referenced this pull request Dec 10, 2024
…45065)

this pr fix 2 things:
1.when first_value/last_value second parameter is true, which means
ignore nulls, fe should not do some rewrite.
2.adding check that first_value/last_value second parameter must be true
or false.

Related PR: #44996 #27623
yiguolei pushed a commit that referenced this pull request Dec 11, 2024
…gnore nulls #45065 (#45236)

Cherry-picked from #45065

Co-authored-by: feiniaofeiafei <[email protected]>
BiteTheDDDDt pushed a commit that referenced this pull request Dec 12, 2024
…44996)

### What problem does this PR solve?

Problem Summary:
needs another prs of FE: #45065
#45264

1. the first value even if have set value, should not return directly,
need check it whether need arg_ignore_null, as maybe it's NULL

2. the last_value if need arg_ignore_null and not set value, should find
in while loop, if find could return directly, not find check iff has any
value before

### Release note

fix first_value/last_value return error with ignore null param is true
github-actions bot pushed a commit that referenced this pull request Dec 12, 2024
…44996)

### What problem does this PR solve?

Problem Summary:
needs another prs of FE: #45065
#45264

1. the first value even if have set value, should not return directly,
need check it whether need arg_ignore_null, as maybe it's NULL

2. the last_value if need arg_ignore_null and not set value, should find
in while loop, if find could return directly, not find check iff has any
value before

### Release note

fix first_value/last_value return error with ignore null param is true
github-actions bot pushed a commit that referenced this pull request Dec 12, 2024
…44996)

### What problem does this PR solve?

Problem Summary:
needs another prs of FE: #45065
#45264

1. the first value even if have set value, should not return directly,
need check it whether need arg_ignore_null, as maybe it's NULL

2. the last_value if need arg_ignore_null and not set value, should find
in while loop, if find could return directly, not find check iff has any
value before

### Release note

fix first_value/last_value return error with ignore null param is true
zhangstar333 pushed a commit that referenced this pull request Dec 16, 2024
…gnore nulls #45065 (#45235)

Cherry-picked from #45065

Co-authored-by: feiniaofeiafei <[email protected]>
yiguolei pushed a commit that referenced this pull request Dec 16, 2024
…44996)

### What problem does this PR solve?

Problem Summary:
needs another prs of FE: #45065
#45264

1. the first value even if have set value, should not return directly,
need check it whether need arg_ignore_null, as maybe it's NULL

2. the last_value if need arg_ignore_null and not set value, should find
in while loop, if find could return directly, not find check iff has any
value before

### Release note

fix first_value/last_value return error with ignore null param is true
zhangstar333 added a commit to zhangstar333/incubator-doris that referenced this pull request Dec 16, 2024
…pache#44996)

### What problem does this PR solve?

Problem Summary:
needs another prs of FE: apache#45065
apache#45264

1. the first value even if have set value, should not return directly,
need check it whether need arg_ignore_null, as maybe it's NULL

2. the last_value if need arg_ignore_null and not set value, should find
in while loop, if find could return directly, not find check iff has any
value before

### Release note

fix first_value/last_value return error with ignore null param is true
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants