Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chore](inverted index) do not print downgrade reason for inverted index try query #45789

Merged
merged 2 commits into from
Dec 23, 2024

Conversation

airborne12
Copy link
Member

@airborne12 airborne12 commented Dec 23, 2024

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Change level of logs below to debug mode

will downgrade without index to evaluate predicate, because of res: [E-6004]hit count: 413, bkd inverted reached limit 50%, segment num rows:33

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@airborne12
Copy link
Member Author

run buildall

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 23, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 40088 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4aadfd18510bce7303e59a1e7d5490835678f5a8, data reload: false

------ Round 1 ----------------------------------
q1	17607	7467	7324	7324
q2	2045	183	175	175
q3	10832	1109	1215	1109
q4	10543	754	729	729
q5	7627	2781	2740	2740
q6	244	150	151	150
q7	995	619	615	615
q8	9248	1933	1911	1911
q9	6749	6525	6392	6392
q10	7078	2320	2332	2320
q11	483	260	256	256
q12	425	224	230	224
q13	17762	2997	3010	2997
q14	245	220	209	209
q15	569	503	500	500
q16	673	596	591	591
q17	998	596	524	524
q18	7364	6673	6821	6673
q19	1352	1042	1067	1042
q20	471	189	186	186
q21	4257	3105	3416	3105
q22	391	321	316	316
Total cold run time: 107958 ms
Total hot run time: 40088 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7299	7301	7301	7301
q2	339	237	236	236
q3	2926	3050	2926	2926
q4	2102	1839	1858	1839
q5	5695	5677	5715	5677
q6	225	145	151	145
q7	2256	1834	1791	1791
q8	3473	3583	3582	3582
q9	8925	8995	8966	8966
q10	3602	3553	3532	3532
q11	619	522	512	512
q12	820	599	579	579
q13	12522	3230	3129	3129
q14	323	290	274	274
q15	576	514	515	514
q16	687	664	652	652
q17	1860	1683	1632	1632
q18	8495	7864	7716	7716
q19	1772	1592	1594	1592
q20	2127	1924	1858	1858
q21	5765	5697	5586	5586
q22	701	589	564	564
Total cold run time: 73109 ms
Total hot run time: 60603 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196473 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4aadfd18510bce7303e59a1e7d5490835678f5a8, data reload: false

query1	1331	992	913	913
query2	6259	2410	2369	2369
query3	10942	4539	4808	4539
query4	32956	23470	23216	23216
query5	3982	473	437	437
query6	301	180	171	171
query7	3985	308	306	306
query8	291	223	244	223
query9	9682	2766	2744	2744
query10	462	245	251	245
query11	17670	15067	15027	15027
query12	146	106	97	97
query13	1565	422	415	415
query14	9181	7265	7336	7265
query15	276	200	189	189
query16	7985	429	491	429
query17	1596	659	587	587
query18	2086	340	339	339
query19	284	163	168	163
query20	132	117	116	116
query21	207	114	114	114
query22	4902	4639	4535	4535
query23	34395	33634	33880	33634
query24	10487	2464	2511	2464
query25	620	385	401	385
query26	1474	161	160	160
query27	2624	336	332	332
query28	7723	2504	2525	2504
query29	926	466	432	432
query30	231	153	149	149
query31	1065	851	822	822
query32	93	59	54	54
query33	762	303	292	292
query34	941	541	520	520
query35	901	774	767	767
query36	1122	958	956	956
query37	145	77	71	71
query38	4424	4296	4108	4108
query39	1539	1451	1465	1451
query40	269	102	106	102
query41	48	46	42	42
query42	113	100	101	100
query43	538	515	504	504
query44	1291	841	820	820
query45	194	176	169	169
query46	1194	726	740	726
query47	2026	1931	1924	1924
query48	416	331	325	325
query49	1029	397	412	397
query50	819	404	389	389
query51	7357	7237	7237	7237
query52	109	91	96	91
query53	267	198	182	182
query54	1260	424	426	424
query55	86	82	90	82
query56	265	248	245	245
query57	1297	1193	1152	1152
query58	248	215	242	215
query59	3407	3317	3396	3317
query60	277	260	244	244
query61	110	102	108	102
query62	926	689	689	689
query63	209	186	186	186
query64	4762	682	642	642
query65	3259	3157	3197	3157
query66	1058	299	296	296
query67	16017	15513	15544	15513
query68	5104	559	562	559
query69	489	260	255	255
query70	1169	1139	1108	1108
query71	454	248	243	243
query72	7012	4038	4082	4038
query73	785	360	373	360
query74	10494	8752	8813	8752
query75	3409	2636	2665	2636
query76	3770	1061	1039	1039
query77	565	266	367	266
query78	10391	9364	9519	9364
query79	1259	614	599	599
query80	636	422	440	422
query81	510	235	228	228
query82	593	121	129	121
query83	171	148	143	143
query84	243	65	74	65
query85	1450	317	349	317
query86	457	281	289	281
query87	4752	4672	4511	4511
query88	3584	2272	2213	2213
query89	423	284	299	284
query90	2037	192	185	185
query91	146	103	103	103
query92	71	52	53	52
query93	2050	560	570	560
query94	875	284	285	284
query95	358	246	248	246
query96	619	292	281	281
query97	2830	2639	2671	2639
query98	217	193	201	193
query99	1713	1314	1319	1314
Total cold run time: 303180 ms
Total hot run time: 196473 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.89% (10134/26056)
Line Coverage: 29.83% (85319/286045)
Region Coverage: 28.94% (43553/150490)
Branch Coverage: 25.46% (22196/87166)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4aadfd18510bce7303e59a1e7d5490835678f5a8_4aadfd18510bce7303e59a1e7d5490835678f5a8/report/index.html

@doris-robot
Copy link

ClickBench: Total hot run time: 33.13 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4aadfd18510bce7303e59a1e7d5490835678f5a8, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.04
query3	0.24	0.08	0.07
query4	1.59	0.10	0.10
query5	0.40	0.41	0.41
query6	1.14	0.67	0.66
query7	0.02	0.01	0.02
query8	0.04	0.04	0.03
query9	0.58	0.51	0.50
query10	0.54	0.57	0.56
query11	0.14	0.11	0.11
query12	0.13	0.11	0.10
query13	0.60	0.61	0.61
query14	2.72	2.83	2.85
query15	0.90	0.82	0.82
query16	0.38	0.38	0.40
query17	1.06	1.05	1.06
query18	0.22	0.21	0.21
query19	1.91	1.82	1.99
query20	0.01	0.01	0.01
query21	15.35	0.58	0.58
query22	2.76	2.75	2.03
query23	17.06	0.93	0.84
query24	2.61	1.08	2.09
query25	0.23	0.27	0.05
query26	0.53	0.14	0.14
query27	0.05	0.04	0.04
query28	10.02	1.12	1.09
query29	12.56	3.23	3.25
query30	0.25	0.06	0.06
query31	2.84	0.38	0.38
query32	3.26	0.47	0.50
query33	3.11	3.17	3.05
query34	16.69	4.49	4.49
query35	4.53	4.48	4.47
query36	0.70	0.48	0.51
query37	0.10	0.06	0.06
query38	0.04	0.03	0.04
query39	0.04	0.02	0.02
query40	0.16	0.14	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 105.76 s
Total hot run time: 33.13 s

Copy link
Contributor

@qidaye qidaye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@airborne12 airborne12 merged commit 2d65c59 into apache:master Dec 23, 2024
27 of 30 checks passed
@airborne12 airborne12 deleted the chore branch December 23, 2024 10:15
github-actions bot pushed a commit that referenced this pull request Dec 23, 2024
…dex try query (#45789)



Change level of logs below to debug mode
```
will downgrade without index to evaluate predicate, because of res: [E-6004]hit count: 413, bkd inverted reached limit 50%, segment num rows:33
```
dataroaring pushed a commit that referenced this pull request Dec 25, 2024
… inverted index try query #45789 (#45814)

Cherry-picked from #45789

Co-authored-by: airborne12 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.4-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants