Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enchement](api)Change the behavior of follower http api redirect to master to follower request master. #45921

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

hubgeter
Copy link
Contributor

What problem does this PR solve?

Problem Summary:
Change the behavior of follower http api redirecting to master to follower requesting master.
Before this, when client sends http api request to follower, some api interfaces will send redirect request to client, and client will send request to master. However, in k8s scenario, there may be network disconnection between client and master, so this pr changes this behavior.

Release note

Change the behavior of follower http api redirecting to master to follower requesting master.

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@hubgeter
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32355 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9d2a2fb7f121b96672c05d4684b7aa13d99b10ee, data reload: false

------ Round 1 ----------------------------------
q1	17633	6301	6026	6026
q2	2045	315	170	170
q3	10494	1209	752	752
q4	10298	880	444	444
q5	8743	2174	1944	1944
q6	207	185	149	149
q7	894	760	604	604
q8	9232	1322	1110	1110
q9	5203	4967	4955	4955
q10	6754	2315	1871	1871
q11	477	276	247	247
q12	351	354	212	212
q13	17761	3615	2918	2918
q14	236	240	211	211
q15	582	503	497	497
q16	641	616	600	600
q17	555	840	314	314
q18	6976	6376	6340	6340
q19	3657	1014	562	562
q20	289	313	181	181
q21	2710	2116	1934	1934
q22	356	324	314	314
Total cold run time: 106094 ms
Total hot run time: 32355 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6429	6198	6226	6198
q2	229	328	225	225
q3	2239	2632	2305	2305
q4	1386	1842	1365	1365
q5	4387	4760	4857	4760
q6	179	174	137	137
q7	2090	1976	1797	1797
q8	2612	2755	2636	2636
q9	7298	7318	7383	7318
q10	3087	3387	2762	2762
q11	569	502	507	502
q12	646	751	645	645
q13	3338	3701	3084	3084
q14	305	310	289	289
q15	561	522	523	522
q16	638	693	627	627
q17	1202	1751	1256	1256
q18	7638	7546	7295	7295
q19	782	1329	1061	1061
q20	2009	2050	1953	1953
q21	5759	5216	4977	4977
q22	620	661	586	586
Total cold run time: 54003 ms
Total hot run time: 52300 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198180 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9d2a2fb7f121b96672c05d4684b7aa13d99b10ee, data reload: false

query1	1308	998	896	896
query2	6494	2454	2344	2344
query3	11086	4782	4933	4782
query4	32951	24101	23562	23562
query5	4101	588	451	451
query6	278	197	185	185
query7	4005	500	323	323
query8	305	248	240	240
query9	9453	2775	2752	2752
query10	441	312	250	250
query11	17935	15328	15222	15222
query12	169	108	110	108
query13	1620	540	412	412
query14	9746	7413	7544	7413
query15	245	209	189	189
query16	7952	613	471	471
query17	1554	763	570	570
query18	2066	414	311	311
query19	218	195	165	165
query20	151	114	115	114
query21	208	128	108	108
query22	4660	4635	4727	4635
query23	34570	33833	34248	33833
query24	6686	2315	2331	2315
query25	466	448	388	388
query26	785	272	159	159
query27	2122	458	333	333
query28	5587	2516	2509	2509
query29	577	563	417	417
query30	209	194	151	151
query31	999	973	820	820
query32	73	60	60	60
query33	491	360	303	303
query34	800	859	529	529
query35	828	843	787	787
query36	1011	1060	972	972
query37	114	100	74	74
query38	4427	4395	4160	4160
query39	1533	1460	1474	1460
query40	209	115	102	102
query41	47	45	49	45
query42	127	105	105	105
query43	506	525	484	484
query44	1379	831	841	831
query45	210	179	178	178
query46	877	1061	667	667
query47	1999	2015	1949	1949
query48	398	425	339	339
query49	755	475	400	400
query50	648	674	398	398
query51	7253	7401	7195	7195
query52	101	104	96	96
query53	230	267	188	188
query54	492	520	412	412
query55	84	79	87	79
query56	260	271	243	243
query57	1301	1234	1170	1170
query58	239	224	238	224
query59	3211	3191	3207	3191
query60	295	271	244	244
query61	109	105	111	105
query62	862	848	759	759
query63	236	195	200	195
query64	3275	1038	653	653
query65	3324	3289	3275	3275
query66	715	401	311	311
query67	16618	15874	15626	15626
query68	9451	751	554	554
query69	470	299	252	252
query70	1183	1163	1120	1120
query71	430	271	244	244
query72	6264	3849	3851	3849
query73	658	753	373	373
query74	10557	9127	9175	9127
query75	4642	3145	2644	2644
query76	5212	1179	762	762
query77	1052	351	270	270
query78	10047	10350	9575	9575
query79	2650	880	601	601
query80	705	508	449	449
query81	477	272	242	242
query82	607	157	123	123
query83	193	163	189	163
query84	275	93	74	74
query85	782	345	300	300
query86	357	304	272	272
query87	4438	4670	4372	4372
query88	3056	2274	2227	2227
query89	417	330	291	291
query90	1817	185	187	185
query91	132	133	111	111
query92	65	52	50	50
query93	1041	889	544	544
query94	656	378	293	293
query95	318	284	254	254
query96	493	591	277	277
query97	2741	2841	2689	2689
query98	220	201	197	197
query99	1731	1559	1459	1459
Total cold run time: 297121 ms
Total hot run time: 198180 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.25 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9d2a2fb7f121b96672c05d4684b7aa13d99b10ee, data reload: false

query1	0.03	0.04	0.03
query2	0.08	0.03	0.03
query3	0.24	0.07	0.06
query4	1.61	0.11	0.11
query5	0.42	0.42	0.41
query6	1.16	0.65	0.65
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.60	0.51	0.52
query10	0.55	0.56	0.56
query11	0.14	0.11	0.11
query12	0.13	0.11	0.11
query13	0.60	0.61	0.60
query14	2.74	2.74	2.76
query15	0.89	0.81	0.84
query16	0.37	0.39	0.38
query17	1.04	1.01	1.03
query18	0.24	0.21	0.21
query19	1.96	1.78	1.98
query20	0.01	0.02	0.01
query21	15.40	0.94	0.58
query22	0.75	0.74	0.60
query23	15.48	1.48	0.56
query24	2.94	1.42	1.05
query25	0.22	0.17	0.11
query26	0.29	0.15	0.15
query27	0.06	0.05	0.05
query28	13.62	1.52	1.05
query29	12.59	3.92	3.23
query30	0.25	0.09	0.06
query31	2.84	0.60	0.37
query32	3.24	0.53	0.46
query33	3.08	3.06	3.11
query34	16.78	5.09	4.49
query35	4.56	4.49	4.55
query36	0.62	0.49	0.50
query37	0.10	0.06	0.05
query38	0.04	0.04	0.03
query39	0.04	0.03	0.02
query40	0.16	0.13	0.12
query41	0.08	0.03	0.02
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.09 s
Total hot run time: 31.25 s

@hubgeter
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32911 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e18e55ccacce66e2fae8f847c7963e67311e217c, data reload: false

------ Round 1 ----------------------------------
q1	17625	6170	6105	6105
q2	2042	307	171	171
q3	10410	1238	750	750
q4	10205	867	432	432
q5	7554	2170	1969	1969
q6	211	183	147	147
q7	919	769	603	603
q8	9230	1358	1151	1151
q9	5185	5062	5017	5017
q10	6766	2322	1894	1894
q11	477	281	257	257
q12	353	365	216	216
q13	20430	3677	2916	2916
q14	235	227	217	217
q15	576	508	509	508
q16	647	627	617	617
q17	552	847	331	331
q18	6937	6596	6548	6548
q19	1768	988	565	565
q20	320	318	183	183
q21	2877	2286	2005	2005
q22	370	346	309	309
Total cold run time: 105689 ms
Total hot run time: 32911 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6319	6258	6248	6248
q2	247	334	237	237
q3	2421	2867	2504	2504
q4	1454	1886	1392	1392
q5	4540	5022	5057	5022
q6	186	178	144	144
q7	2182	1999	1839	1839
q8	2661	2791	2708	2708
q9	7444	7326	7382	7326
q10	3079	3338	2751	2751
q11	567	530	495	495
q12	689	791	624	624
q13	3361	3731	3131	3131
q14	305	318	298	298
q15	576	516	513	513
q16	652	689	684	684
q17	1202	1709	1284	1284
q18	7796	7556	7437	7437
q19	851	1213	1074	1074
q20	1975	2044	1946	1946
q21	5616	5307	4940	4940
q22	636	622	565	565
Total cold run time: 54759 ms
Total hot run time: 53162 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196931 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e18e55ccacce66e2fae8f847c7963e67311e217c, data reload: false

query1	1306	971	1002	971
query2	6502	2231	2252	2231
query3	11005	4901	4431	4431
query4	33085	24007	23255	23255
query5	4843	647	468	468
query6	296	201	182	182
query7	3981	488	317	317
query8	308	276	236	236
query9	9395	2742	2743	2742
query10	466	299	259	259
query11	18589	15405	15106	15106
query12	161	103	99	99
query13	1637	562	418	418
query14	11504	7618	7205	7205
query15	255	214	181	181
query16	7918	574	465	465
query17	1477	742	572	572
query18	2159	429	342	342
query19	218	193	160	160
query20	116	115	109	109
query21	208	127	103	103
query22	4610	4640	4688	4640
query23	35495	33705	33914	33705
query24	6423	2437	2378	2378
query25	460	446	383	383
query26	789	251	161	161
query27	2272	467	359	359
query28	5643	2505	2482	2482
query29	552	524	433	433
query30	207	180	149	149
query31	984	937	840	840
query32	74	56	54	54
query33	489	364	298	298
query34	750	876	529	529
query35	819	844	801	801
query36	1038	1042	975	975
query37	117	101	77	77
query38	4377	4402	4234	4234
query39	1554	1489	1537	1489
query40	213	121	104	104
query41	44	46	50	46
query42	128	111	104	104
query43	508	525	488	488
query44	1392	841	835	835
query45	195	183	171	171
query46	913	1067	666	666
query47	2024	1973	1954	1954
query48	398	403	337	337
query49	712	495	388	388
query50	636	664	404	404
query51	7465	7138	7308	7138
query52	107	103	91	91
query53	222	256	187	187
query54	499	527	433	433
query55	84	80	76	76
query56	259	269	258	258
query57	1261	1244	1179	1179
query58	242	216	228	216
query59	3159	3286	3084	3084
query60	288	276	265	265
query61	138	128	130	128
query62	887	825	771	771
query63	236	198	203	198
query64	3685	1140	769	769
query65	3379	3289	3271	3271
query66	836	414	312	312
query67	16603	15864	15565	15565
query68	10007	774	538	538
query69	488	299	253	253
query70	1277	1160	1166	1160
query71	439	279	263	263
query72	5835	3972	3579	3579
query73	2260	759	357	357
query74	10100	9157	9075	9075
query75	4888	3116	2691	2691
query76	5763	1191	770	770
query77	1044	371	274	274
query78	10243	10361	9449	9449
query79	2922	848	593	593
query80	732	504	433	433
query81	491	263	236	236
query82	478	196	129	129
query83	186	159	137	137
query84	280	94	68	68
query85	758	367	297	297
query86	355	306	303	303
query87	4566	4465	4405	4405
query88	3343	2286	2223	2223
query89	431	338	298	298
query90	1988	185	185	185
query91	131	132	173	132
query92	67	53	50	50
query93	1560	934	542	542
query94	662	391	283	283
query95	327	275	248	248
query96	493	604	293	293
query97	2731	2874	2704	2704
query98	223	202	187	187
query99	1634	1520	1499	1499
Total cold run time: 305297 ms
Total hot run time: 196931 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.36 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e18e55ccacce66e2fae8f847c7963e67311e217c, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.03	0.04
query3	0.24	0.07	0.06
query4	1.62	0.11	0.11
query5	0.41	0.41	0.42
query6	1.15	0.66	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.50	0.52
query10	0.55	0.59	0.55
query11	0.15	0.10	0.11
query12	0.14	0.11	0.10
query13	0.62	0.61	0.60
query14	2.72	2.71	2.84
query15	0.89	0.83	0.83
query16	0.39	0.36	0.38
query17	1.02	0.99	1.06
query18	0.23	0.21	0.21
query19	1.93	1.81	1.94
query20	0.01	0.00	0.02
query21	15.36	0.94	0.56
query22	0.77	0.82	0.67
query23	15.25	1.47	0.53
query24	3.56	1.10	1.30
query25	0.19	0.06	0.09
query26	0.22	0.15	0.14
query27	0.06	0.06	0.04
query28	14.04	1.52	1.05
query29	12.61	3.95	3.27
query30	0.26	0.09	0.07
query31	2.82	0.62	0.39
query32	3.24	0.55	0.47
query33	3.05	3.17	3.14
query34	16.99	5.16	4.50
query35	4.48	4.46	4.49
query36	0.66	0.49	0.50
query37	0.09	0.07	0.06
query38	0.05	0.04	0.03
query39	0.03	0.03	0.02
query40	0.17	0.13	0.13
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.87 s
Total hot run time: 31.36 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants