Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[pick](branch-3.0) pick #41310 #41969 #44945

Merged
merged 4 commits into from
Dec 6, 2024
Merged

[pick](branch-3.0) pick #41310 #41969 #44945

merged 4 commits into from
Dec 6, 2024

Conversation

xinyiZzz
Copy link
Contributor

@xinyiZzz xinyiZzz commented Dec 3, 2024

What problem does this PR solve?

pick #41310 #41969

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@xinyiZzz
[opt](memory) Support Memory Profile (apache#41310)
63ea477 
1. Support Memory Tracker to create Memory Profile snapshots.
2. Remove BE `mem_tracker` web page and replace it with more intuitive
Memory Profile.
3. All memory logs also print more intuitive Memory Profile.
@xinyiZzz
[fix](memory) Process Profile web page support copy to clipboard (apa…
5e90812 
…che#41969)

TODO, expect more information about process status, CPU, IO, etc.

![image](https://github.com/user-attachments/assets/14b10e39-e42a-46a4-ac61-c689e55d7396)

![image](https://github.com/user-attachments/assets/5514b0e3-29a9-4f97-9ebd-d83d972493bf)
@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Dec 3, 2024

run buildall

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Dec 3, 2024

run buildall

2 similar comments
@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Dec 4, 2024

run buildall

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Dec 4, 2024

run buildall

@xinyiZzz xinyiZzz force-pushed the branch-3.0_20241204_pick branch from fdcbc25 to fda14df Compare December 4, 2024 07:43
@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Dec 4, 2024

run buildall

@xinyiZzz xinyiZzz force-pushed the branch-3.0_20241204_pick branch from fda14df to 63088a0 Compare December 4, 2024 07:47
@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Dec 4, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40412 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 63088a0ed86fc6b19f864bef6af680d091a10380, data reload: false

------ Round 1 ----------------------------------
q1	17569	7415	7322	7322
q2	2075	174	175	174
q3	10651	1049	1145	1049
q4	10571	718	769	718
q5	7759	2743	2832	2743
q6	237	155	148	148
q7	959	603	590	590
q8	9580	1869	2010	1869
q9	6623	6384	6455	6384
q10	6990	2253	2320	2253
q11	460	268	263	263
q12	404	208	209	208
q13	17772	2936	2967	2936
q14	251	209	217	209
q15	568	520	508	508
q16	685	591	622	591
q17	975	535	576	535
q18	7165	6658	6613	6613
q19	1372	971	992	971
q20	483	205	200	200
q21	3856	3251	3162	3162
q22	1053	970	966	966
Total cold run time: 108058 ms
Total hot run time: 40412 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7351	7229	7247	7229
q2	327	223	222	222
q3	2917	2837	2831	2831
q4	2020	1785	1775	1775
q5	5660	5695	5719	5695
q6	230	141	142	141
q7	2217	1743	1785	1743
q8	3341	3523	3534	3523
q9	8673	8878	8839	8839
q10	3533	3497	3513	3497
q11	592	513	506	506
q12	790	585	587	585
q13	16517	3177	3146	3146
q14	303	269	272	269
q15	562	523	524	523
q16	711	672	687	672
q17	1893	1614	1627	1614
q18	8184	7737	7597	7597
q19	1758	1628	1628	1628
q20	2039	1865	1860	1860
q21	5363	5273	5239	5239
q22	1127	1020	1051	1020
Total cold run time: 76108 ms
Total hot run time: 60154 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 195850 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 63088a0ed86fc6b19f864bef6af680d091a10380, data reload: false

query1	1270	914	921	914
query2	6231	2154	2062	2062
query3	10836	4101	3912	3912
query4	67064	30077	23614	23614
query5	5151	433	423	423
query6	415	180	179	179
query7	5636	307	308	307
query8	302	226	225	225
query9	9357	2712	2690	2690
query10	484	267	279	267
query11	17638	15406	15761	15406
query12	162	97	100	97
query13	1563	431	429	429
query14	10858	6951	7582	6951
query15	214	182	174	174
query16	7044	530	516	516
query17	1081	587	598	587
query18	1928	330	307	307
query19	201	161	159	159
query20	119	117	112	112
query21	210	105	105	105
query22	4743	4614	4357	4357
query23	34850	34245	33996	33996
query24	6101	2908	2894	2894
query25	504	387	399	387
query26	666	170	166	166
query27	1697	307	301	301
query28	4372	2529	2546	2529
query29	668	425	425	425
query30	233	168	164	164
query31	990	844	836	836
query32	63	58	51	51
query33	454	271	282	271
query34	887	502	501	501
query35	834	704	725	704
query36	1088	963	965	963
query37	113	70	75	70
query38	4081	4039	3984	3984
query39	1531	1484	1477	1477
query40	210	94	94	94
query41	46	47	47	47
query42	109	97	94	94
query43	542	505	501	501
query44	1196	806	816	806
query45	186	170	168	168
query46	1135	731	730	730
query47	2011	1921	1893	1893
query48	496	382	374	374
query49	761	406	393	393
query50	843	422	407	407
query51	7271	7198	7134	7134
query52	95	88	89	88
query53	254	179	186	179
query54	566	461	459	459
query55	74	75	75	75
query56	255	235	241	235
query57	1209	1087	1137	1087
query58	229	206	229	206
query59	3112	3030	2969	2969
query60	279	267	254	254
query61	140	125	126	125
query62	762	664	677	664
query63	215	185	197	185
query64	1534	744	609	609
query65	3245	3154	3117	3117
query66	727	301	299	299
query67	15775	15320	15375	15320
query68	4527	538	542	538
query69	444	254	255	254
query70	1153	1129	1136	1129
query71	406	250	241	241
query72	6252	3937	3906	3906
query73	752	339	338	338
query74	10064	8879	8972	8879
query75	3344	2552	2607	2552
query76	2215	1038	1072	1038
query77	559	264	265	264
query78	10808	9669	9412	9412
query79	9189	586	575	575
query80	2302	417	409	409
query81	541	242	234	234
query82	1376	116	113	113
query83	257	145	144	144
query84	291	76	82	76
query85	1745	308	283	283
query86	482	288	295	288
query87	4500	4202	4337	4202
query88	5629	2358	2380	2358
query89	548	282	286	282
query90	2077	177	176	176
query91	180	144	144	144
query92	72	48	47	47
query93	7033	530	539	530
query94	907	275	285	275
query95	348	241	237	237
query96	613	278	279	278
query97	3311	3135	3155	3135
query98	226	206	192	192
query99	1772	1313	1297	1297
Total cold run time: 338839 ms
Total hot run time: 195850 ms

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Dec 5, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40511 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9ed934910be8164005b3e70929879e135b52048c, data reload: false

------ Round 1 ----------------------------------
q1	17585	7450	7325	7325
q2	2064	170	197	170
q3	10853	1045	1168	1045
q4	10564	757	770	757
q5	7744	2827	2800	2800
q6	239	143	144	143
q7	965	602	577	577
q8	9349	1900	2034	1900
q9	6613	6428	6405	6405
q10	6995	2257	2287	2257
q11	463	265	254	254
q12	405	211	206	206
q13	17753	2977	3014	2977
q14	233	207	224	207
q15	557	517	514	514
q16	673	614	611	611
q17	968	532	553	532
q18	7263	6622	6486	6486
q19	2202	1068	1034	1034
q20	482	202	192	192
q21	3973	3112	3151	3112
q22	1079	1010	1007	1007
Total cold run time: 109022 ms
Total hot run time: 40511 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7332	7232	7739	7232
q2	323	237	227	227
q3	2939	2921	2878	2878
q4	2032	1815	1768	1768
q5	5682	5697	5722	5697
q6	222	149	140	140
q7	2208	1803	1765	1765
q8	3322	3515	3422	3422
q9	8874	8877	8815	8815
q10	3577	3529	3528	3528
q11	603	503	503	503
q12	803	655	585	585
q13	16428	3177	3147	3147
q14	303	268	266	266
q15	559	537	521	521
q16	703	664	674	664
q17	1836	1583	1563	1563
q18	8148	8028	7566	7566
q19	3075	1518	1580	1518
q20	2070	1845	1862	1845
q21	5482	5307	5118	5118
q22	1128	994	1008	994
Total cold run time: 77649 ms
Total hot run time: 59762 ms

@yiguolei yiguolei merged commit ff7c8cb into apache:branch-3.0 Dec 6, 2024
17 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@xinyiZzz @doris-robot @yiguolei