Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](inverted index) Fix Null Pointer Exception in function match (#45456) #45775

Closed
wants to merge 1 commit into from
Closed

[fix](inverted index) Fix Null Pointer Exception in function match (#45456) #45775

wants to merge 1 commit into from

Conversation

zzzxl1993
Copy link
Contributor

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zzzxl1993
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49125 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c75de83baadfdd75ddccc45ea7f136d9b34415df, data reload: false

------ Round 1 ----------------------------------
q1	17924	4356	4438	4356
q2	2059	155	152	152
q3	10344	1892	1931	1892
q4	10367	1233	1320	1233
q5	8438	4226	3939	3939
q6	237	124	120	120
q7	2051	1608	1624	1608
q8	9498	2726	2718	2718
q9	13169	9931	9950	9931
q10	8672	3512	3521	3512
q11	412	244	235	235
q12	457	296	298	296
q13	18364	4007	4057	4007
q14	344	328	319	319
q15	508	458	462	458
q16	521	457	455	455
q17	1132	931	956	931
q18	7285	6736	6944	6736
q19	1695	1550	1433	1433
q20	543	317	288	288
q21	4470	4138	4130	4130
q22	486	376	413	376
Total cold run time: 118976 ms
Total hot run time: 49125 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4294	4330	4258	4258
q2	324	227	225	225
q3	4190	4111	4111	4111
q4	2737	2752	2740	2740
q5	7178	7080	7136	7080
q6	239	118	117	117
q7	3239	2806	2798	2798
q8	4336	4449	4445	4445
q9	14272	13802	13894	13802
q10	4247	4252	4301	4252
q11	719	684	666	666
q12	1029	841	867	841
q13	6291	3790	3762	3762
q14	456	432	418	418
q15	502	455	434	434
q16	622	586	596	586
q17	3777	3862	3789	3789
q18	8724	8815	8750	8750
q19	1738	1621	1661	1621
q20	2393	2150	2146	2146
q21	8497	8460	8481	8460
q22	983	962	962	962
Total cold run time: 80787 ms
Total hot run time: 76263 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 213784 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c75de83baadfdd75ddccc45ea7f136d9b34415df, data reload: false

query1	936	421	384	384
query2	8867	2082	2171	2082
query3	8099	203	209	203
query4	24617	21684	21426	21426
query5	19751	6489	6509	6489
query6	326	226	234	226
query7	4384	298	302	298
query8	261	245	227	227
query9	3088	2670	2578	2578
query10	434	297	315	297
query11	15907	14911	15063	14911
query12	129	75	74	74
query13	1038	449	446	446
query14	18015	13789	13896	13789
query15	401	214	228	214
query16	6439	273	276	273
query17	1747	949	891	891
query18	895	321	311	311
query19	209	150	144	144
query20	96	94	94	94
query21	193	98	95	95
query22	5234	5162	5048	5048
query23	34035	33412	33231	33231
query24	6648	6356	6293	6293
query25	541	456	416	416
query26	1008	170	163	163
query27	2352	305	296	296
query28	6119	2253	2253	2253
query29	2793	2761	2903	2761
query30	245	170	165	165
query31	962	779	764	764
query32	71	58	58	58
query33	448	266	252	252
query34	863	475	480	475
query35	1140	949	940	940
query36	1354	1282	1254	1254
query37	92	59	60	59
query38	3049	2898	2965	2898
query39	1380	1317	1313	1313
query40	203	102	96	96
query41	40	38	37	37
query42	86	83	81	81
query43	667	601	604	601
query44	1208	722	720	720
query45	243	230	230	230
query46	1233	935	953	935
query47	1893	1914	1714	1714
query48	510	409	413	409
query49	630	374	402	374
query50	882	657	609	609
query51	4730	4666	4651	4651
query52	81	73	84	73
query53	220	182	193	182
query54	2674	2432	2474	2432
query55	85	87	73	73
query56	227	219	202	202
query57	1212	1144	1163	1144
query58	217	216	216	216
query59	3404	3376	3376	3376
query60	216	202	203	202
query61	100	108	121	108
query62	879	470	454	454
query63	201	176	177	176
query64	3367	1566	1523	1523
query65	3638	3566	3538	3538
query66	793	421	390	390
query67	17347	16131	18676	16131
query68	7082	661	653	653
query69	481	271	285	271
query70	1422	1505	1478	1478
query71	405	302	311	302
query72	7044	4664	4946	4664
query73	748	313	316	313
query74	6225	5838	5873	5838
query75	4648	3735	3716	3716
query76	4032	1177	1211	1177
query77	559	259	258	258
query78	12719	11729	11603	11603
query79	5907	665	635	635
query80	2733	390	397	390
query81	532	239	240	239
query82	1637	94	94	94
query83	222	137	130	130
query84	260	69	74	69
query85	1393	330	326	326
query86	366	294	277	277
query87	3257	3079	2972	2972
query88	4735	2323	2332	2323
query89	357	306	284	284
query90	1745	214	214	214
query91	160	126	139	126
query92	60	52	52	52
query93	3378	584	556	556
query94	904	219	212	212
query95	2061	2000	2021	2000
query96	637	321	335	321
query97	6548	6414	6450	6414
query98	226	206	205	205
query99	2929	915	904	904
Total cold run time: 316564 ms
Total hot run time: 213784 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.82 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c75de83baadfdd75ddccc45ea7f136d9b34415df, data reload: false

query1	0.02	0.03	0.03
query2	0.07	0.02	0.02
query3	0.25	0.05	0.04
query4	1.77	0.06	0.06
query5	0.54	0.52	0.52
query6	1.34	0.65	0.61
query7	0.02	0.01	0.01
query8	0.03	0.02	0.03
query9	0.52	0.48	0.47
query10	0.53	0.52	0.53
query11	0.13	0.08	0.08
query12	0.13	0.09	0.09
query13	0.62	0.62	0.60
query14	0.77	0.79	0.79
query15	0.77	0.76	0.76
query16	0.36	0.36	0.35
query17	0.97	1.04	0.97
query18	0.21	0.25	0.24
query19	1.92	1.84	1.84
query20	0.02	0.01	0.01
query21	15.49	0.57	0.55
query22	2.43	2.03	1.64
query23	16.88	1.03	1.10
query24	7.00	1.54	0.97
query25	0.35	0.10	0.06
query26	0.81	0.14	0.17
query27	0.04	0.05	0.05
query28	5.64	0.79	0.71
query29	12.61	2.30	2.28
query30	0.56	0.51	0.52
query31	2.81	0.39	0.36
query32	3.37	0.50	0.50
query33	3.08	3.05	3.05
query34	15.25	4.81	4.80
query35	4.90	4.86	4.86
query36	1.06	1.02	1.02
query37	0.06	0.05	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.07	0.02	0.01
query42	0.02	0.01	0.01
query43	0.03	0.02	0.02
Total cold run time: 103.66 s
Total hot run time: 30.82 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit c75de83baadfdd75ddccc45ea7f136d9b34415df with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       22.0 seconds inserted 10000000 Rows, about 454K ops/s

@zzzxl1993
[fix](inverted index) Fix Null Pointer Exception in function match (a…
94635a3 
…pache#45456)

Problem Summary:

Using select match_phrase('', '') in the old optimizer causes issues
@zzzxl1993 zzzxl1993 force-pushed the branch-2.0.202412161434 branch from c75de83 to 94635a3 Compare December 23, 2024 07:57
@zzzxl1993
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.58% (8316/21555)
Line Coverage: 30.27% (68758/227178)
Region Coverage: 29.70% (35411/119246)
Branch Coverage: 25.46% (18208/71528)
Coverage Report: http://coverage.selectdb-in.cc/coverage/94635a33978aa9f101cbbb22d6c8b9ba6fd5522f_94635a33978aa9f101cbbb22d6c8b9ba6fd5522f/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 49535 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 94635a33978aa9f101cbbb22d6c8b9ba6fd5522f, data reload: false

------ Round 1 ----------------------------------
q1	17928	4464	4417	4417
q2	2069	161	148	148
q3	10324	1925	1941	1925
q4	10099	1256	1319	1256
q5	8501	3885	4275	3885
q6	230	123	125	123
q7	2088	1607	1620	1607
q8	9513	2780	2752	2752
q9	10898	10029	9879	9879
q10	8709	3583	3546	3546
q11	426	247	241	241
q12	467	292	295	292
q13	18325	4050	4038	4038
q14	361	323	343	323
q15	506	465	463	463
q16	548	459	474	459
q17	1169	1001	965	965
q18	7241	6802	7008	6802
q19	1707	1604	1560	1560
q20	525	321	313	313
q21	4534	4146	4148	4146
q22	502	395	403	395
Total cold run time: 116670 ms
Total hot run time: 49535 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4369	4335	4381	4335
q2	332	240	221	221
q3	4181	4147	4177	4147
q4	2749	2756	2759	2756
q5	7227	7212	7158	7158
q6	242	126	121	121
q7	3239	2927	2838	2838
q8	4405	4489	4588	4489
q9	13717	13695	13594	13594
q10	4290	4252	4266	4252
q11	754	676	658	658
q12	1019	852	874	852
q13	7015	3785	3745	3745
q14	441	425	416	416
q15	498	463	459	459
q16	651	592	578	578
q17	3841	3873	3761	3761
q18	8818	8743	8861	8743
q19	1749	1699	1724	1699
q20	2366	2130	2107	2107
q21	8543	8522	8452	8452
q22	1048	906	945	906
Total cold run time: 81494 ms
Total hot run time: 76287 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 215070 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 94635a33978aa9f101cbbb22d6c8b9ba6fd5522f, data reload: false

query1	947	428	388	388
query2	6555	2308	2366	2308
query3	6924	203	205	203
query4	23540	21797	21464	21464
query5	19728	6555	6559	6555
query6	292	222	233	222
query7	4353	302	315	302
query8	264	252	253	252
query9	3074	2675	2583	2583
query10	473	302	303	302
query11	15671	15494	15056	15056
query12	132	79	79	79
query13	1038	457	452	452
query14	17080	13346	13392	13346
query15	387	222	231	222
query16	5927	274	259	259
query17	1770	941	917	917
query18	899	321	315	315
query19	221	154	147	147
query20	113	101	97	97
query21	195	95	99	95
query22	5234	4978	4952	4952
query23	34151	33553	33582	33553
query24	7937	6383	6413	6383
query25	533	429	415	415
query26	1340	166	156	156
query27	2425	290	293	290
query28	6063	2302	2263	2263
query29	2959	2781	2823	2781
query30	244	173	162	162
query31	944	730	779	730
query32	74	61	58	58
query33	451	249	268	249
query34	860	492	468	468
query35	1143	928	924	924
query36	1182	1237	1127	1127
query37	177	61	63	61
query38	3078	2940	2938	2938
query39	1375	1306	1309	1306
query40	312	94	96	94
query41	41	37	38	37
query42	90	97	89	89
query43	724	616	607	607
query44	1236	719	715	715
query45	242	238	230	230
query46	1235	942	932	932
query47	1729	1943	1742	1742
query48	507	414	418	414
query49	645	372	396	372
query50	873	623	614	614
query51	4812	4724	4751	4724
query52	104	81	74	74
query53	228	181	188	181
query54	2693	2452	2483	2452
query55	85	83	82	82
query56	218	212	205	205
query57	1221	1159	1284	1159
query58	226	212	200	200
query59	3601	3515	3259	3259
query60	219	210	220	210
query61	100	95	99	95
query62	851	573	476	476
query63	201	180	172	172
query64	3558	1547	1353	1353
query65	3693	3593	3551	3551
query66	823	413	439	413
query67	16031	18028	17515	17515
query68	6276	649	635	635
query69	480	282	277	277
query70	1636	1400	1518	1400
query71	376	316	308	308
query72	6596	4819	4833	4819
query73	735	328	328	328
query74	6243	5835	5905	5835
query75	4309	3731	3666	3666
query76	3122	1157	1214	1157
query77	436	250	264	250
query78	12613	11636	11518	11518
query79	5707	665	626	626
query80	3322	404	385	385
query81	543	244	238	238
query82	1618	106	97	97
query83	328	137	130	130
query84	259	69	69	69
query85	1609	316	310	310
query86	435	332	287	287
query87	3251	3023	3005	3005
query88	4786	2292	2327	2292
query89	342	287	271	271
query90	1643	209	212	209
query91	157	134	135	134
query92	72	52	52	52
query93	3333	588	559	559
query94	936	209	211	209
query95	1973	1961	1930	1930
query96	630	325	319	319
query97	6526	6331	6429	6331
query98	229	216	220	216
query99	2895	893	861	861
Total cold run time: 309467 ms
Total hot run time: 215070 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.95 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 94635a33978aa9f101cbbb22d6c8b9ba6fd5522f, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.03	0.02
query3	0.24	0.05	0.05
query4	1.77	0.08	0.07
query5	0.54	0.52	0.52
query6	1.23	0.62	0.66
query7	0.02	0.00	0.01
query8	0.03	0.02	0.02
query9	0.50	0.49	0.49
query10	0.55	0.53	0.54
query11	0.12	0.08	0.08
query12	0.12	0.09	0.09
query13	0.63	0.61	0.61
query14	0.78	0.79	0.78
query15	0.78	0.76	0.76
query16	0.37	0.37	0.38
query17	1.01	1.01	1.03
query18	0.24	0.26	0.24
query19	1.91	1.84	1.84
query20	0.02	0.01	0.01
query21	15.48	0.57	0.57
query22	2.06	2.19	1.59
query23	17.14	1.14	0.93
query24	5.99	1.10	1.06
query25	0.33	0.10	0.05
query26	0.73	0.15	0.15
query27	0.05	0.03	0.04
query28	6.88	0.74	0.72
query29	12.62	2.37	2.37
query30	0.59	0.52	0.52
query31	2.82	0.38	0.37
query32	3.40	0.50	0.50
query33	3.06	3.06	3.07
query34	15.25	4.80	4.77
query35	4.82	4.87	4.85
query36	1.04	1.02	1.02
query37	0.06	0.05	0.05
query38	0.04	0.02	0.02
query39	0.03	0.02	0.01
query40	0.17	0.14	0.14
query41	0.06	0.01	0.02
query42	0.03	0.02	0.01
query43	0.03	0.02	0.02
Total cold run time: 103.64 s
Total hot run time: 30.95 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 94635a33978aa9f101cbbb22d6c8b9ba6fd5522f with default session variables
Stream load json:         21 seconds loaded 2358488459 Bytes, about 107 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       22.1 seconds inserted 10000000 Rows, about 452K ops/s

@zzzxl1993 zzzxl1993 closed this Dec 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@zzzxl1993 @hello-stephen @doris-robot