Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[chore](rename) Forbid renaming partition columns since this is a buggy feature #47596

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[chore](rename) Forbid renaming partition columns since this is a buggy feature #47596

wants to merge 1 commit into from
+83 −24

Conversation

TangSiyang2001
Copy link
Collaborator

What problem does this PR solve

Problem Summary:

Rename partition columns should also rename column names in partition expressions, but this is not implemented currently. Therefore, forbid renaming partition columns temporarily.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Feb 7, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@TangSiyang2001
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31756 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 55b759eb04784755d53c1ec12b60b482b567c475, data reload: false

------ Round 1 ----------------------------------
q1	17571	5318	5139	5139
q2	2048	325	175	175
q3	10373	1312	745	745
q4	10217	1018	555	555
q5	7515	2426	2344	2344
q6	193	171	135	135
q7	904	749	615	615
q8	9305	1256	1103	1103
q9	5114	4691	4601	4601
q10	6832	2326	1927	1927
q11	487	291	266	266
q12	350	353	222	222
q13	17759	3701	3103	3103
q14	223	223	206	206
q15	511	478	460	460
q16	624	615	581	581
q17	571	887	339	339
q18	6532	6214	6201	6201
q19	1217	965	540	540
q20	319	328	197	197
q21	2938	2277	1979	1979
q22	374	334	323	323
Total cold run time: 101977 ms
Total hot run time: 31756 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5161	5216	5299	5216
q2	240	327	237	237
q3	2185	2674	2260	2260
q4	1470	1933	1366	1366
q5	4242	4132	4178	4132
q6	211	161	123	123
q7	1901	1810	1800	1800
q8	2621	2572	2558	2558
q9	7342	7154	7089	7089
q10	2985	3247	2778	2778
q11	601	516	490	490
q12	690	800	608	608
q13	3669	3971	3255	3255
q14	277	292	268	268
q15	505	470	473	470
q16	646	673	641	641
q17	1157	1636	1306	1306
q18	7634	7420	7221	7221
q19	812	770	812	770
q20	1999	2029	1882	1882
q21	5409	4932	4828	4828
q22	632	623	570	570
Total cold run time: 52389 ms
Total hot run time: 49868 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 184836 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 55b759eb04784755d53c1ec12b60b482b567c475, data reload: false

query1	982	409	380	380
query2	6553	1812	1821	1812
query3	6810	216	216	216
query4	26390	24208	23347	23347
query5	4310	674	508	508
query6	296	206	187	187
query7	4604	515	301	301
query8	284	254	228	228
query9	8601	2516	2510	2510
query10	467	313	261	261
query11	15476	15102	14910	14910
query12	153	109	122	109
query13	1671	535	389	389
query14	8768	6798	6778	6778
query15	211	187	183	183
query16	7386	638	469	469
query17	942	721	578	578
query18	1967	404	309	309
query19	204	194	162	162
query20	123	123	125	123
query21	217	129	107	107
query22	4220	4316	4054	4054
query23	33849	33023	32936	32936
query24	7738	2460	2424	2424
query25	540	448	393	393
query26	1229	274	160	160
query27	2155	498	334	334
query28	3922	2407	2381	2381
query29	757	549	465	465
query30	232	188	153	153
query31	913	880	809	809
query32	76	84	62	62
query33	552	359	301	301
query34	792	854	504	504
query35	819	819	754	754
query36	981	977	901	901
query37	124	96	77	77
query38	4199	4184	4211	4184
query39	1446	1427	1384	1384
query40	207	116	106	106
query41	53	51	50	50
query42	126	111	111	111
query43	497	516	481	481
query44	1322	784	779	779
query45	177	181	164	164
query46	909	1053	658	658
query47	1715	1757	1718	1718
query48	375	418	300	300
query49	786	520	441	441
query50	728	772	432	432
query51	4227	4174	4091	4091
query52	104	108	99	99
query53	233	263	185	185
query54	500	480	408	408
query55	85	84	84	84
query56	253	261	261	261
query57	1108	1141	1119	1119
query58	253	238	257	238
query59	2505	2550	2554	2550
query60	283	283	269	269
query61	116	117	115	115
query62	806	721	674	674
query63	222	191	192	191
query64	4335	1047	647	647
query65	3281	3153	3165	3153
query66	1160	405	314	314
query67	15978	15689	15742	15689
query68	6787	791	522	522
query69	481	308	270	270
query70	1175	1128	1109	1109
query71	392	314	274	274
query72	5735	3580	3820	3580
query73	738	736	351	351
query74	9226	9307	9020	9020
query75	3155	3166	2734	2734
query76	3173	1204	789	789
query77	473	380	284	284
query78	9896	10012	9294	9294
query79	2705	834	600	600
query80	664	524	462	462
query81	499	273	243	243
query82	494	154	118	118
query83	183	178	229	178
query84	297	98	80	80
query85	814	351	308	308
query86	375	306	272	272
query87	4422	4576	4448	4448
query88	3938	2182	2171	2171
query89	398	324	292	292
query90	1872	200	198	198
query91	139	233	109	109
query92	82	63	58	58
query93	2366	1042	577	577
query94	704	412	304	304
query95	358	272	275	272
query96	491	571	266	266
query97	2741	2842	2702	2702
query98	231	206	201	201
query99	1349	1415	1280	1280
Total cold run time: 269656 ms
Total hot run time: 184836 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.44 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 55b759eb04784755d53c1ec12b60b482b567c475, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.03
query3	0.24	0.07	0.07
query4	1.62	0.10	0.10
query5	0.43	0.43	0.39
query6	1.16	0.67	0.66
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.53	0.52
query10	0.58	0.58	0.58
query11	0.16	0.11	0.11
query12	0.14	0.11	0.11
query13	0.62	0.60	0.60
query14	2.71	2.82	2.82
query15	0.92	0.86	0.86
query16	0.39	0.36	0.38
query17	1.06	1.02	1.02
query18	0.21	0.20	0.19
query19	1.98	1.76	1.98
query20	0.02	0.02	0.01
query21	15.34	0.90	0.55
query22	0.76	1.27	0.68
query23	14.81	1.40	0.64
query24	7.21	2.23	0.53
query25	0.56	0.19	0.09
query26	0.58	0.16	0.15
query27	0.06	0.05	0.05
query28	10.27	0.82	0.43
query29	12.60	4.02	3.27
query30	0.24	0.09	0.06
query31	2.82	0.59	0.38
query32	3.22	0.57	0.47
query33	3.00	3.05	3.04
query34	15.82	5.12	4.51
query35	4.55	4.52	4.52
query36	0.66	0.51	0.48
query37	0.09	0.06	0.06
query38	0.06	0.04	0.03
query39	0.02	0.02	0.02
query40	0.17	0.14	0.13
query41	0.09	0.03	0.03
query42	0.03	0.03	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.99 s
Total hot run time: 30.44 s

@TangSiyang2001 TangSiyang2001 force-pushed the forbid-rename-partition-column branch from 55b759e to ec778b5 Compare February 7, 2025 09:34
@TangSiyang2001 TangSiyang2001 changed the title [chore](rename) Forbid renaming partition columns due to this is a buggy feature [chore](rename) Forbid renaming partition columns since this is a buggy feature Feb 7, 2025
@TangSiyang2001
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31917 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ec778b5ca4994bd11751d9c6828e7f4866a68705, data reload: false

------ Round 1 ----------------------------------
q1	17609	5269	5175	5175
q2	2050	309	181	181
q3	10575	1276	750	750
q4	10218	1026	540	540
q5	7528	2385	2354	2354
q6	188	166	132	132
q7	892	741	606	606
q8	9307	1256	1168	1168
q9	5108	4783	4890	4783
q10	6819	2319	1911	1911
q11	478	281	253	253
q12	346	351	211	211
q13	17779	3700	3056	3056
q14	230	237	215	215
q15	526	457	468	457
q16	631	614	599	599
q17	585	882	356	356
q18	6743	6161	6263	6161
q19	1924	985	578	578
q20	306	316	188	188
q21	2819	2176	1942	1942
q22	361	330	301	301
Total cold run time: 103022 ms
Total hot run time: 31917 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5232	5176	5183	5176
q2	237	335	231	231
q3	2192	2698	2289	2289
q4	1512	1878	1416	1416
q5	4254	4129	4173	4129
q6	209	165	127	127
q7	1895	1846	1795	1795
q8	2629	2656	2568	2568
q9	7362	7228	7257	7228
q10	3024	3275	2813	2813
q11	577	498	504	498
q12	711	783	612	612
q13	3366	3927	3251	3251
q14	295	288	276	276
q15	528	460	473	460
q16	670	666	628	628
q17	1136	1587	1393	1393
q18	7672	7456	7328	7328
q19	819	845	1003	845
q20	1976	2002	1938	1938
q21	5586	5020	4694	4694
q22	618	602	552	552
Total cold run time: 52500 ms
Total hot run time: 50247 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189974 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ec778b5ca4994bd11751d9c6828e7f4866a68705, data reload: false

query1	1327	953	932	932
query2	6309	1874	1878	1874
query3	11016	4572	4458	4458
query4	54816	24807	23017	23017
query5	5261	547	485	485
query6	372	196	186	186
query7	4987	485	290	290
query8	314	245	231	231
query9	5883	2489	2508	2489
query10	419	305	272	272
query11	15113	14997	14935	14935
query12	163	110	104	104
query13	1093	512	381	381
query14	10645	6795	6804	6795
query15	208	200	191	191
query16	7037	664	488	488
query17	1061	728	562	562
query18	1533	414	304	304
query19	195	201	158	158
query20	132	121	119	119
query21	211	128	108	108
query22	4431	4455	4295	4295
query23	34076	33370	33515	33370
query24	5774	2455	2468	2455
query25	482	478	429	429
query26	677	285	158	158
query27	1761	506	340	340
query28	2739	2489	2417	2417
query29	566	571	426	426
query30	224	189	154	154
query31	880	849	831	831
query32	75	61	58	58
query33	445	376	315	315
query34	764	881	532	532
query35	793	858	737	737
query36	976	977	911	911
query37	130	101	77	77
query38	4285	4196	4272	4196
query39	1506	1440	1605	1440
query40	214	119	103	103
query41	50	50	47	47
query42	122	107	102	102
query43	515	513	480	480
query44	1282	800	798	798
query45	178	179	166	166
query46	895	1075	677	677
query47	1866	1881	1766	1766
query48	393	420	317	317
query49	683	537	422	422
query50	740	762	422	422
query51	4212	4374	4203	4203
query52	109	108	98	98
query53	238	267	188	188
query54	476	506	411	411
query55	84	80	84	80
query56	263	268	272	268
query57	1155	1209	1113	1113
query58	247	238	257	238
query59	2675	2975	2676	2676
query60	283	260	268	260
query61	155	124	114	114
query62	766	748	684	684
query63	222	196	186	186
query64	2088	1015	657	657
query65	3225	3178	3138	3138
query66	776	393	305	305
query67	15936	15537	15420	15420
query68	5595	786	515	515
query69	529	312	252	252
query70	1164	1117	1081	1081
query71	444	282	250	250
query72	6000	3654	3849	3654
query73	1239	775	349	349
query74	8879	9179	8931	8931
query75	3205	3148	2684	2684
query76	3932	1168	746	746
query77	548	366	277	277
query78	10196	10369	9327	9327
query79	2049	844	600	600
query80	832	521	458	458
query81	534	268	238	238
query82	426	155	118	118
query83	184	164	150	150
query84	290	103	75	75
query85	757	356	298	298
query86	389	294	281	281
query87	4489	4457	4432	4432
query88	2740	2200	2173	2173
query89	386	311	280	280
query90	1608	189	194	189
query91	134	139	105	105
query92	65	59	55	55
query93	2476	1009	571	571
query94	661	415	296	296
query95	351	266	259	259
query96	489	555	267	267
query97	2748	2876	2708	2708
query98	229	203	208	203
query99	1332	1393	1293	1293
Total cold run time: 293427 ms
Total hot run time: 189974 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.42 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ec778b5ca4994bd11751d9c6828e7f4866a68705, data reload: false

query1	0.04	0.03	0.02
query2	0.09	0.03	0.04
query3	0.23	0.07	0.07
query4	1.62	0.09	0.10
query5	0.41	0.41	0.39
query6	1.16	0.65	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.60	0.52	0.54
query10	0.58	0.58	0.57
query11	0.15	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.60
query14	2.67	2.79	2.72
query15	0.93	0.87	0.85
query16	0.38	0.39	0.37
query17	1.05	1.02	1.03
query18	0.22	0.20	0.20
query19	1.93	1.83	2.01
query20	0.01	0.02	0.01
query21	15.36	0.89	0.55
query22	0.77	1.22	0.67
query23	14.90	1.40	0.61
query24	7.42	1.13	0.65
query25	0.52	0.33	0.06
query26	0.60	0.16	0.13
query27	0.05	0.04	0.05
query28	9.27	0.91	0.43
query29	12.54	4.00	3.29
query30	0.25	0.08	0.06
query31	2.82	0.59	0.38
query32	3.23	0.55	0.48
query33	2.96	3.00	3.02
query34	15.69	5.14	4.53
query35	4.60	4.57	4.55
query36	0.65	0.50	0.48
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.02	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.03
query43	0.04	0.03	0.02
Total cold run time: 105 s
Total hot run time: 30.42 s

gavinchou
gavinchou reviewed Feb 7, 2025
View reviewed changes
fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
// but this is not implemented currently. Therefore, forbid renaming partition columns temporarily.
PartitionInfo partitionInfo = table.getPartitionInfo();
if (partitionInfo.getPartitionColumns().stream().anyMatch(c -> c.getName().equalsIgnoreCase(colName))) {
throw new DdlException("Renaming partition columns has problems, forbidden in current Doris version");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LOG detail "Renaming partition columns has problems, forbidden in current Doris version"
and throw a simple msg "renaming partition columns is unsupported"

dataroaring
dataroaring approved these changes Feb 8, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 8, 2025
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Feb 8, 2025

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Feb 8, 2025

PR approved by anyone and no changes requested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gavinchou gavinchou gavinchou left review comments

@dataroaring dataroaring dataroaring approved these changes

@CalvinKirs CalvinKirs Awaiting requested review from CalvinKirs CalvinKirs is a code owner

@morningman morningman Awaiting requested review from morningman morningman is a code owner

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. kind/behavior-changed reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@TangSiyang2001 @Thearas @doris-robot @gavinchou @dataroaring