[fix](auth) fix use database stmt access unauthorized catalog #45720

vinlee19 · 2024-12-20T07:24:28Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
For example, suppose there is a database named "testdb" in the internal catalog. Meanwhile, there is also a database with the same name in the external catalog. However, you only have the right to access the internal catalog. When you use the "external_catalog.testdb" statement, there is a possibility that you can access the external catalog as well.
Perhaps the issue only has a minimal effect on users.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2024-12-20T07:24:32Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

vinlee19 · 2024-12-20T07:27:33Z

run buildall

doris-robot · 2024-12-20T08:39:20Z

TPC-H: Total hot run time: 39733 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a550f1545bc21527e2a96cb2b84c25fd49f6a5e6, data reload: false

------ Round 1 ----------------------------------
q1	17572	7498	7280	7280
q2	2051	172	167	167
q3	10640	1069	1250	1069
q4	10576	769	778	769
q5	7591	2729	2723	2723
q6	240	151	146	146
q7	992	619	604	604
q8	9244	1889	1929	1889
q9	6652	6377	6398	6377
q10	7050	2353	2323	2323
q11	468	270	260	260
q12	425	226	227	226
q13	17774	2910	2904	2904
q14	255	214	211	211
q15	557	513	495	495
q16	672	603	578	578
q17	980	527	621	527
q18	7192	6712	6567	6567
q19	1327	1005	985	985
q20	462	184	185	184
q21	3996	3271	3137	3137
q22	383	320	312	312
Total cold run time: 107099 ms
Total hot run time: 39733 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7227	7261	7209	7209
q2	327	229	230	229
q3	2915	2734	2932	2734
q4	2074	1842	1767	1767
q5	5624	5688	5660	5660
q6	228	147	142	142
q7	2261	1772	1787	1772
q8	3342	3581	3507	3507
q9	8915	8976	8951	8951
q10	3615	3535	3527	3527
q11	608	512	527	512
q12	824	620	595	595
q13	11891	3172	3091	3091
q14	301	289	286	286
q15	557	516	516	516
q16	675	617	656	617
q17	1862	1649	1589	1589
q18	8186	7809	7859	7809
q19	1707	1518	1612	1518
q20	2098	1872	1888	1872
q21	5627	5450	5317	5317
q22	642	608	589	589
Total cold run time: 71506 ms
Total hot run time: 59809 ms

doris-robot · 2024-12-20T08:51:38Z

TPC-DS: Total hot run time: 196348 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a550f1545bc21527e2a96cb2b84c25fd49f6a5e6, data reload: false

query1	1301	973	912	912
query2	6222	2342	2399	2342
query3	11116	4772	4839	4772
query4	33579	23246	23485	23246
query5	4960	452	452	452
query6	293	186	189	186
query7	3987	299	311	299
query8	285	229	235	229
query9	9505	2699	2696	2696
query10	466	248	234	234
query11	17887	15025	15092	15025
query12	160	104	102	102
query13	1595	414	397	397
query14	9999	7578	7407	7407
query15	273	194	185	185
query16	8297	488	474	474
query17	1688	606	605	605
query18	2161	312	338	312
query19	376	172	168	168
query20	124	129	123	123
query21	212	109	112	109
query22	4724	4364	4430	4364
query23	35112	33622	33750	33622
query24	10253	2533	2475	2475
query25	570	419	406	406
query26	1106	151	159	151
query27	2503	335	332	332
query28	7417	2450	2434	2434
query29	682	416	415	415
query30	231	144	147	144
query31	1020	860	849	849
query32	94	57	72	57
query33	787	307	289	289
query34	1138	530	522	522
query35	891	768	780	768
query36	1112	967	988	967
query37	131	74	74	74
query38	4254	4201	4354	4201
query39	1523	1470	1450	1450
query40	205	101	100	100
query41	45	40	42	40
query42	112	102	107	102
query43	538	475	488	475
query44	1273	831	815	815
query45	200	164	165	164
query46	1208	739	727	727
query47	2035	1927	1924	1924
query48	417	313	318	313
query49	892	392	383	383
query50	829	408	414	408
query51	7381	7084	7114	7084
query52	103	90	90	90
query53	255	190	188	188
query54	1173	425	427	425
query55	90	78	79	78
query56	265	249	239	239
query57	1282	1178	1138	1138
query58	246	237	222	222
query59	3366	3159	3222	3159
query60	272	266	246	246
query61	110	110	107	107
query62	874	709	685	685
query63	222	191	196	191
query64	3930	695	660	660
query65	3276	3308	3287	3287
query66	777	309	316	309
query67	16490	15541	15533	15533
query68	6028	539	547	539
query69	494	253	316	253
query70	1193	1112	1127	1112
query71	530	238	251	238
query72	6960	4107	4122	4107
query73	782	356	375	356
query74	10289	8904	8780	8780
query75	3834	2609	2761	2609
query76	3841	1180	1050	1050
query77	677	267	267	267
query78	10228	9343	9308	9308
query79	1580	589	596	589
query80	1193	419	424	419
query81	519	228	227	227
query82	366	123	117	117
query83	245	145	146	145
query84	281	73	69	69
query85	991	310	302	302
query86	364	300	299	299
query87	4454	4410	4328	4328
query88	3537	2236	2195	2195
query89	415	298	286	286
query90	2196	183	184	183
query91	141	104	108	104
query92	65	50	59	50
query93	2018	543	543	543
query94	870	296	289	289
query95	349	252	244	244
query96	615	289	273	273
query97	2887	2690	2674	2674
query98	216	198	194	194
query99	1592	1329	1312	1312
Total cold run time: 305650 ms
Total hot run time: 196348 ms

doris-robot · 2024-12-20T08:57:08Z

ClickBench: Total hot run time: 32.67 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a550f1545bc21527e2a96cb2b84c25fd49f6a5e6, data reload: false

query1	0.03	0.04	0.03
query2	0.07	0.03	0.03
query3	0.23	0.08	0.07
query4	1.60	0.11	0.10
query5	0.42	0.39	0.39
query6	1.16	0.66	0.64
query7	0.02	0.01	0.01
query8	0.04	0.04	0.03
query9	0.59	0.50	0.51
query10	0.55	0.56	0.56
query11	0.14	0.10	0.10
query12	0.14	0.11	0.10
query13	0.60	0.60	0.59
query14	2.73	2.84	2.78
query15	0.89	0.83	0.83
query16	0.38	0.38	0.38
query17	1.06	1.06	1.04
query18	0.22	0.21	0.21
query19	1.91	1.84	2.02
query20	0.01	0.01	0.02
query21	15.41	0.57	0.57
query22	2.24	2.01	2.38
query23	17.16	0.88	0.87
query24	3.49	1.26	0.52
query25	0.18	0.23	0.19
query26	0.35	0.13	0.13
query27	0.05	0.04	0.06
query28	10.61	1.10	1.06
query29	12.59	3.28	3.27
query30	0.25	0.06	0.07
query31	2.84	0.39	0.37
query32	3.25	0.46	0.45
query33	3.04	3.10	3.11
query34	17.05	4.51	4.54
query35	4.52	4.51	4.47
query36	0.65	0.50	0.48
query37	0.10	0.06	0.06
query38	0.04	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.13	0.13
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.97 s
Total hot run time: 32.67 s

regression-test/suites/external_table_p0/nereids_commands/test_use_database_stmt.groovy

github-actions · 2024-12-20T09:16:47Z

PR approved by anyone and no changes requested.

vinlee19 · 2024-12-23T05:09:34Z

run buildall

doris-robot · 2024-12-23T06:08:55Z

TPC-H: Total hot run time: 41113 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dd5f7c1a801ce9b5bc8994072ba8f31fddc306f0, data reload: false

------ Round 1 ----------------------------------
q1	17618	7505	7349	7349
q2	2061	1293	1237	1237
q3	10034	1178	1187	1178
q4	10228	784	769	769
q5	7574	2740	2598	2598
q6	243	148	145	145
q7	995	615	616	615
q8	9248	1908	1948	1908
q9	6659	6494	6423	6423
q10	6948	2360	2314	2314
q11	488	272	256	256
q12	439	229	221	221
q13	17758	2922	2991	2922
q14	241	208	202	202
q15	567	510	508	508
q16	656	576	581	576
q17	995	598	539	539
q18	7374	6768	6518	6518
q19	1380	1015	1023	1015
q20	462	191	183	183
q21	3997	3325	3339	3325
q22	368	314	312	312
Total cold run time: 106333 ms
Total hot run time: 41113 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7289	7235	7264	7235
q2	328	230	231	230
q3	2896	2950	2994	2950
q4	2133	1896	1867	1867
q5	5744	5506	5665	5506
q6	227	145	148	145
q7	2235	1815	1789	1789
q8	3409	3515	3514	3514
q9	8963	9053	9122	9053
q10	3600	3594	3618	3594
q11	602	495	495	495
q12	807	606	607	606
q13	12068	3168	3067	3067
q14	330	269	284	269
q15	572	502	519	502
q16	697	637	631	631
q17	1840	1616	1661	1616
q18	8106	7797	7627	7627
q19	3212	1610	1399	1399
q20	2097	1848	1863	1848
q21	5653	5677	5492	5492
q22	635	567	553	553
Total cold run time: 73443 ms
Total hot run time: 59988 ms

doris-robot · 2024-12-23T06:21:20Z

TPC-DS: Total hot run time: 196757 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dd5f7c1a801ce9b5bc8994072ba8f31fddc306f0, data reload: false

query1	1223	823	795	795
query2	6237	2574	2313	2313
query3	11133	4856	5004	4856
query4	33870	23411	23604	23411
query5	4691	491	468	468
query6	282	195	180	180
query7	3992	312	318	312
query8	320	251	242	242
query9	9648	2711	2707	2707
query10	510	263	255	255
query11	17949	15236	15403	15236
query12	168	114	107	107
query13	1623	441	420	420
query14	10072	7514	8003	7514
query15	261	200	193	193
query16	8112	463	488	463
query17	2145	621	593	593
query18	2196	315	317	315
query19	368	165	158	158
query20	127	115	111	111
query21	208	104	103	103
query22	4940	4728	4587	4587
query23	34819	33787	34641	33787
query24	11848	2508	2532	2508
query25	642	406	398	398
query26	1964	161	156	156
query27	2674	346	328	328
query28	8437	2448	2460	2448
query29	1014	421	427	421
query30	235	155	158	155
query31	1035	811	848	811
query32	96	57	57	57
query33	762	311	319	311
query34	1002	550	508	508
query35	867	763	797	763
query36	1144	946	967	946
query37	257	81	89	81
query38	4404	4286	4077	4077
query39	1529	1486	1450	1450
query40	288	110	102	102
query41	45	43	42	42
query42	118	110	102	102
query43	551	500	511	500
query44	1290	818	803	803
query45	183	172	178	172
query46	1173	722	737	722
query47	1912	1814	1839	1814
query48	421	329	312	312
query49	999	382	381	381
query50	809	395	398	395
query51	7126	7144	6979	6979
query52	105	99	91	91
query53	262	181	190	181
query54	947	413	407	407
query55	81	78	82	78
query56	259	229	236	229
query57	1256	1078	1111	1078
query58	232	219	229	219
query59	3280	3107	3257	3107
query60	274	264	255	255
query61	103	104	108	104
query62	837	689	676	676
query63	235	198	200	198
query64	4928	683	649	649
query65	3308	3182	3219	3182
query66	1139	301	302	301
query67	15960	15448	15415	15415
query68	5831	541	554	541
query69	466	251	253	251
query70	1178	1144	1152	1144
query71	428	285	252	252
query72	6639	4180	4093	4093
query73	791	361	363	361
query74	10089	8824	8783	8783
query75	3424	2614	2645	2614
query76	3839	1161	1017	1017
query77	616	296	376	296
query78	10134	9458	9375	9375
query79	1172	608	616	608
query80	822	466	441	441
query81	520	240	234	234
query82	384	123	125	123
query83	264	150	146	146
query84	248	73	78	73
query85	1177	300	305	300
query86	430	306	302	302
query87	4668	4425	4640	4425
query88	4176	2220	2175	2175
query89	423	292	292	292
query90	2129	192	190	190
query91	144	106	106	106
query92	67	52	55	52
query93	2190	561	549	549
query94	868	310	303	303
query95	356	258	259	258
query96	640	279	281	279
query97	2802	2702	2622	2622
query98	226	199	195	195
query99	1553	1325	1319	1319
Total cold run time: 309619 ms
Total hot run time: 196757 ms

doris-robot · 2024-12-23T06:26:51Z

ClickBench: Total hot run time: 32.28 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dd5f7c1a801ce9b5bc8994072ba8f31fddc306f0, data reload: false

query1	0.04	0.04	0.04
query2	0.06	0.04	0.03
query3	0.23	0.07	0.08
query4	1.61	0.11	0.11
query5	0.46	0.40	0.40
query6	1.17	0.66	0.66
query7	0.02	0.02	0.02
query8	0.04	0.03	0.04
query9	0.56	0.52	0.50
query10	0.55	0.58	0.56
query11	0.14	0.10	0.10
query12	0.14	0.11	0.10
query13	0.62	0.60	0.61
query14	2.80	2.74	2.90
query15	0.90	0.82	0.83
query16	0.38	0.38	0.39
query17	1.06	0.99	1.03
query18	0.23	0.22	0.21
query19	1.94	1.82	2.05
query20	0.01	0.01	0.01
query21	15.35	0.56	0.58
query22	3.29	3.05	1.35
query23	17.03	1.05	0.88
query24	3.29	1.07	1.70
query25	0.32	0.09	0.24
query26	0.36	0.13	0.14
query27	0.05	0.04	0.04
query28	9.88	1.11	1.09
query29	12.58	3.20	3.19
query30	0.25	0.07	0.06
query31	2.86	0.39	0.37
query32	3.25	0.46	0.45
query33	3.12	3.07	3.22
query34	16.78	4.47	4.54
query35	4.55	4.50	4.51
query36	0.66	0.48	0.50
query37	0.09	0.05	0.06
query38	0.05	0.03	0.04
query39	0.03	0.02	0.02
query40	0.17	0.12	0.12
query41	0.09	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 107.09 s
Total hot run time: 32.28 s

vinlee19 · 2024-12-23T11:28:42Z

run p0

github-actions · 2024-12-26T02:41:29Z

PR approved by at least one committer and no changes requested.

Problem Summary: For example, suppose there is a database named "testdb" in the internal catalog. Meanwhile, there is also a database with the same name in the external catalog. However, you only have the right to access the internal catalog. When you use the "external_catalog.testdb" statement, there is a possibility that you can access the external catalog as well. Perhaps the issue only has a minimal effect on users.

…alog #45720 (#45978) Cherry-picked from #45720 Co-authored-by: Petrichor <[email protected]>

…alog #45720 (#45977) Cherry-picked from #45720 Co-authored-by: Petrichor <[email protected]>

vinlee19 added 3 commits December 20, 2024 14:03

fix the unauthorized access to the catalog

76c4ebf

add regreesion test

5b6b30c

revert regression-conf

a550f15

vinlee19 changed the title ~~[fix](auth) fix use database stmt access unauthorized access catalog~~ [fix](auth) fix use database stmt access unauthorized catalog Dec 20, 2024

zddr approved these changes Dec 20, 2024

View reviewed changes

regression-test/suites/external_table_p0/nereids_commands/test_use_database_stmt.groovy Outdated Show resolved Hide resolved

github-actions bot added the reviewed label Dec 20, 2024

normalize regression test

dd5f7c1

LiBinfeng-01 approved these changes Dec 24, 2024

View reviewed changes

starocean999 approved these changes Dec 26, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 26, 2024

starocean999 added dev/2.0.x dev/2.1.x dev/3.0.x labels Dec 26, 2024

starocean999 merged commit 5c97856 into apache:master Dec 26, 2024
28 checks passed

github-actions bot mentioned this pull request Dec 26, 2024

branch-3.0: [fix](auth) fix use database stmt access unauthorized catalog #45720 #45977

Merged

github-actions bot mentioned this pull request Dec 26, 2024

branch-2.1: [fix](auth) fix use database stmt access unauthorized catalog #45720 #45978

Merged

yiguolei pushed a commit that referenced this pull request Dec 30, 2024

branch-2.1: [fix](auth) fix use database stmt access unauthorized cat…

5425e46

…alog #45720 (#45978) Cherry-picked from #45720 Co-authored-by: Petrichor <[email protected]>

yiguolei added dev/2.1.8-merged and removed dev/2.1.x labels Dec 30, 2024

yiguolei pushed a commit that referenced this pull request Dec 30, 2024

branch-3.0: [fix](auth) fix use database stmt access unauthorized cat…

f43ac38

…alog #45720 (#45977) Cherry-picked from #45720 Co-authored-by: Petrichor <[email protected]>

yiguolei added dev/3.0.4-merged and removed dev/3.0.x labels Dec 30, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[fix](auth) fix use database stmt access unauthorized catalog #45720

[fix](auth) fix use database stmt access unauthorized catalog #45720

vinlee19 commented Dec 20, 2024

Thearas commented Dec 20, 2024

vinlee19 commented Dec 20, 2024

doris-robot commented Dec 20, 2024

doris-robot commented Dec 20, 2024

doris-robot commented Dec 20, 2024

github-actions bot commented Dec 20, 2024

vinlee19 commented Dec 23, 2024

doris-robot commented Dec 23, 2024

doris-robot commented Dec 23, 2024

doris-robot commented Dec 23, 2024

vinlee19 commented Dec 23, 2024

github-actions bot commented Dec 26, 2024

[fix](auth) fix use database stmt access unauthorized catalog #45720

[fix](auth) fix use database stmt access unauthorized catalog #45720

Conversation

vinlee19 commented Dec 20, 2024

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Thearas commented Dec 20, 2024

vinlee19 commented Dec 20, 2024

doris-robot commented Dec 20, 2024

doris-robot commented Dec 20, 2024

doris-robot commented Dec 20, 2024

github-actions bot commented Dec 20, 2024

vinlee19 commented Dec 23, 2024

doris-robot commented Dec 23, 2024

doris-robot commented Dec 23, 2024

doris-robot commented Dec 23, 2024

vinlee19 commented Dec 23, 2024

github-actions bot commented Dec 26, 2024