Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](olap) Set the original tablet state to TABLET_SHUTDOWN #45812

Merged
merged 3 commits into from
Dec 25, 2024
Merged

[fix](olap) Set the original tablet state to TABLET_SHUTDOWN #45812

merged 3 commits into from
Dec 25, 2024

Conversation

w41ter
Copy link
Contributor

@w41ter w41ter commented Dec 23, 2024 •
edited
Loading

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Set the original tablet state to TABLET_SHUTDOWN when loading a new tablet from the disk during the restore job. Otherwise, the other thread may hold the old tablet object, and save meta too.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Dec 23, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@w41ter
Copy link
Contributor Author

w41ter commented Dec 23, 2024

run p0

@w41ter
Copy link
Contributor Author

w41ter commented Dec 23, 2024

run compile

@w41ter
[fix](olap) Set the original tablet state to TABLET_SHUTDOWN
e190eee 
when loading new tablet from disk during restore. Otherwise the other thread may
hold the old tablet object, and save meta too.
@w41ter w41ter force-pushed the fix_check_table_size_correctness branch from dda4093 to e190eee Compare December 24, 2024 02:26
@w41ter
Copy link
Contributor Author

w41ter commented Dec 24, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.80% (10094/26015)
Line Coverage: 29.79% (85153/285866)
Region Coverage: 28.90% (43464/150403)
Branch Coverage: 25.44% (22159/87116)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e190eee937e76a86099f69f1cad918196ee25e8c_e190eee937e76a86099f69f1cad918196ee25e8c/report/index.html

@w41ter w41ter changed the title Add log [fix](olap) Set the original tablet state to TABLET_SHUTDOWN Dec 24, 2024
@w41ter w41ter marked this pull request as ready for review December 24, 2024 02:59
@w41ter
Copy link
Contributor Author

w41ter commented Dec 24, 2024

run buildall

@w41ter
Copy link
Contributor Author

w41ter commented Dec 24, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40062 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 55c1374f738c01a41e47dfaedf5e542114db9159, data reload: false

------ Round 1 ----------------------------------
q1	17599	7512	7352	7352
q2	2046	178	170	170
q3	10549	1140	1176	1140
q4	10225	766	765	765
q5	7591	2680	2760	2680
q6	244	151	152	151
q7	989	610	646	610
q8	9255	1919	1942	1919
q9	6816	6412	6450	6412
q10	7047	2318	2359	2318
q11	470	264	251	251
q12	568	229	227	227
q13	17788	2989	2903	2903
q14	237	216	205	205
q15	543	507	504	504
q16	658	582	588	582
q17	993	548	515	515
q18	7479	6730	6707	6707
q19	1336	993	1040	993
q20	463	183	190	183
q21	4060	3202	3163	3163
q22	383	312	326	312
Total cold run time: 107339 ms
Total hot run time: 40062 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7279	7282	7250	7250
q2	337	230	230	230
q3	2907	2805	2791	2791
q4	2005	1717	1672	1672
q5	5371	5370	5439	5370
q6	219	142	144	142
q7	2154	1754	1757	1754
q8	3271	3420	3431	3420
q9	8640	8618	8579	8579
q10	3480	3435	3472	3435
q11	586	510	495	495
q12	792	616	604	604
q13	16363	3014	2990	2990
q14	283	256	277	256
q15	563	496	492	492
q16	679	626	659	626
q17	1820	1609	1558	1558
q18	7788	7545	7311	7311
q19	1687	1479	1529	1479
q20	2053	1847	1872	1847
q21	5493	5385	5223	5223
q22	631	590	601	590
Total cold run time: 74401 ms
Total hot run time: 58114 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.80% (10095/26015)
Line Coverage: 29.79% (85149/285866)
Region Coverage: 28.91% (43484/150403)
Branch Coverage: 25.44% (22163/87116)
Coverage Report: http://coverage.selectdb-in.cc/coverage/55c1374f738c01a41e47dfaedf5e542114db9159_55c1374f738c01a41e47dfaedf5e542114db9159/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 189598 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 55c1374f738c01a41e47dfaedf5e542114db9159, data reload: false

query1	879	394	357	357
query2	3434	2349	2281	2281
query3	5550	216	218	216
query4	33266	23356	23510	23356
query5	2805	486	481	481
query6	261	182	195	182
query7	4087	306	319	306
query8	308	238	236	236
query9	9308	2768	2768	2768
query10	444	247	247	247
query11	18145	15136	15172	15136
query12	147	109	103	103
query13	1477	406	417	406
query14	9163	7576	6812	6812
query15	284	173	183	173
query16	7094	452	416	416
query17	1047	559	559	559
query18	1959	293	322	293
query19	357	149	151	149
query20	115	107	116	107
query21	212	106	123	106
query22	4555	4370	4118	4118
query23	34566	33422	33974	33422
query24	11184	2584	2483	2483
query25	672	378	381	378
query26	1903	156	159	156
query27	2964	340	336	336
query28	7284	2469	2459	2459
query29	954	425	407	407
query30	277	157	155	155
query31	1029	778	823	778
query32	71	55	59	55
query33	716	298	300	298
query34	969	522	517	517
query35	878	729	741	729
query36	1082	947	937	937
query37	269	80	79	79
query38	4068	4226	4141	4141
query39	1497	1399	1410	1399
query40	290	101	103	101
query41	46	43	52	43
query42	116	100	99	99
query43	531	492	486	486
query44	1217	804	817	804
query45	181	164	162	162
query46	1173	706	700	700
query47	1914	1816	1832	1816
query48	415	318	333	318
query49	885	384	378	378
query50	804	377	395	377
query51	7253	6942	7169	6942
query52	109	98	93	93
query53	257	183	183	183
query54	971	404	408	404
query55	87	77	79	77
query56	285	255	256	255
query57	1287	1098	1105	1098
query58	231	225	233	225
query59	3287	3209	3065	3065
query60	274	252	249	249
query61	113	103	106	103
query62	902	656	684	656
query63	215	187	196	187
query64	4223	740	635	635
query65	3306	3209	3233	3209
query66	1002	312	317	312
query67	15703	15520	15431	15431
query68	5191	591	558	558
query69	419	300	253	253
query70	1244	1051	1072	1051
query71	338	250	262	250
query72	5970	4159	4060	4060
query73	781	367	368	367
query74	10063	8853	8943	8853
query75	3433	2626	2645	2626
query76	2455	1079	1072	1072
query77	473	281	276	276
query78	10152	9398	9358	9358
query79	1067	617	613	613
query80	691	462	480	462
query81	499	236	232	232
query82	229	121	120	120
query83	198	160	159	159
query84	259	165	66	66
query85	954	311	300	300
query86	321	300	298	298
query87	4664	4372	4321	4321
query88	3538	2245	2222	2222
query89	387	300	298	298
query90	2055	189	189	189
query91	157	104	107	104
query92	63	52	55	52
query93	1060	548	546	546
query94	882	281	279	279
query95	361	250	251	250
query96	700	279	286	279
query97	2810	2627	2681	2627
query98	220	195	196	195
query99	1528	1317	1341	1317
Total cold run time: 285519 ms
Total hot run time: 189598 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.85 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 55c1374f738c01a41e47dfaedf5e542114db9159, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.62	0.11	0.10
query5	0.43	0.40	0.42
query6	1.17	0.64	0.65
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.59	0.51	0.53
query10	0.55	0.58	0.55
query11	0.14	0.11	0.11
query12	0.15	0.11	0.11
query13	0.60	0.60	0.59
query14	2.82	2.88	2.74
query15	0.90	0.81	0.82
query16	0.39	0.38	0.38
query17	1.00	1.05	1.02
query18	0.24	0.21	0.21
query19	1.97	1.78	2.05
query20	0.02	0.00	0.02
query21	15.36	0.60	0.58
query22	2.60	2.05	2.44
query23	16.80	1.04	0.87
query24	3.40	1.48	1.00
query25	0.13	0.17	0.09
query26	0.61	0.15	0.14
query27	0.05	0.05	0.04
query28	10.17	1.09	1.07
query29	12.61	3.28	3.27
query30	0.25	0.06	0.06
query31	2.86	0.39	0.38
query32	3.23	0.48	0.47
query33	3.07	3.08	3.21
query34	16.85	4.43	4.45
query35	4.51	4.40	4.47
query36	0.66	0.48	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 106.65 s
Total hot run time: 32.85 s

@w41ter w41ter closed this Dec 24, 2024
@w41ter w41ter reopened this Dec 24, 2024
dataroaring
dataroaring approved these changes Dec 24, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 24, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@dataroaring dataroaring merged commit 190c12d into apache:master Dec 25, 2024
35 of 39 checks passed
github-actions bot pushed a commit that referenced this pull request Dec 25, 2024
@w41ter
[fix](olap) Set the original tablet state to TABLET_SHUTDOWN (#45812)
d05e66e 
Set the original tablet state to TABLET_SHUTDOWN when loading a new
tablet from the disk during the restore job. Otherwise, the other thread
may hold the old tablet object, and save meta too.
@github-actions github-actions bot mentioned this pull request Dec 25, 2024
Merged
@w41ter w41ter deleted the fix_check_table_size_correctness branch December 25, 2024 02:20
dataroaring pushed a commit that referenced this pull request Dec 26, 2024
@github-actions @w41ter
branch-3.0: [fix](olap) Set the original tablet state to TABLET_SHUTD…
3a1743a 
…OWN #45812 (#45891)

Cherry-picked from #45812

Co-authored-by: walter <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dataroaring dataroaring dataroaring approved these changes

@Yukang-Lian Yukang-Lian Yukang-Lian approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.4-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@w41ter @Thearas @doris-robot @Yukang-Lian @dataroaring