Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

branch-3.0: [chore](io) Add debug log for critical file operations #46770 #46837

Merged
merged 1 commit into from
Jan 13, 2025
Merged

branch-3.0: [chore](io) Add debug log for critical file operations #46770 #46837

merged 1 commit into from
Jan 13, 2025

Conversation

github-actions[bot]
Copy link
Contributor

Cherry-picked from #46770

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Jan 12, 2025
@hello-stephen
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41043 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c4aa22f0ce11c33d48ba76811d4e299c79eff994, data reload: false

------ Round 1 ----------------------------------
q1	17636	7443	7278	7278
q2	2045	185	173	173
q3	10850	1091	1164	1091
q4	10554	751	716	716
q5	7755	2833	2831	2831
q6	238	147	141	141
q7	982	617	605	605
q8	9346	1956	2041	1956
q9	6581	6513	6487	6487
q10	7041	2310	2301	2301
q11	463	293	265	265
q12	416	211	208	208
q13	17796	2995	3019	2995
q14	256	215	210	210
q15	561	524	530	524
q16	688	611	635	611
q17	982	555	612	555
q18	7177	6772	6665	6665
q19	1456	1028	1073	1028
q20	467	223	206	206
q21	4083	3218	3360	3218
q22	1082	1001	979	979
Total cold run time: 108455 ms
Total hot run time: 41043 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7318	7237	7271	7237
q2	327	235	245	235
q3	2961	2972	2973	2972
q4	2026	1805	1807	1805
q5	5715	5723	5703	5703
q6	223	143	148	143
q7	2253	1818	1842	1818
q8	3332	3574	3523	3523
q9	8801	8845	8837	8837
q10	3568	3534	3525	3525
q11	599	502	505	502
q12	800	580	589	580
q13	8611	3170	3210	3170
q14	301	291	274	274
q15	573	543	511	511
q16	709	673	664	664
q17	1851	1599	1576	1576
q18	8242	7679	7656	7656
q19	1669	1548	1552	1548
q20	2112	1889	1912	1889
q21	5680	5246	5419	5246
q22	1160	1045	1027	1027
Total cold run time: 68831 ms
Total hot run time: 60441 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197247 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c4aa22f0ce11c33d48ba76811d4e299c79eff994, data reload: false

query1	1342	933	904	904
query2	6219	2109	2095	2095
query3	10813	4228	4120	4120
query4	65974	29212	23487	23487
query5	4972	441	445	441
query6	411	172	179	172
query7	5546	316	318	316
query8	302	234	237	234
query9	8915	2681	2648	2648
query10	441	261	252	252
query11	17045	15261	15946	15261
query12	165	100	105	100
query13	1471	461	447	447
query14	10395	7177	7491	7177
query15	205	182	184	182
query16	7272	466	477	466
query17	1082	592	596	592
query18	2067	338	340	338
query19	225	168	180	168
query20	128	112	117	112
query21	207	105	115	105
query22	4777	4423	4448	4423
query23	34870	33984	34558	33984
query24	6125	2938	2907	2907
query25	536	424	458	424
query26	660	174	176	174
query27	1760	366	375	366
query28	4264	2481	2452	2452
query29	711	499	440	440
query30	248	165	161	161
query31	1004	846	844	844
query32	65	54	54	54
query33	421	291	295	291
query34	904	510	509	509
query35	878	730	730	730
query36	1096	981	986	981
query37	128	74	70	70
query38	4208	3991	4057	3991
query39	1505	1483	1464	1464
query40	203	108	98	98
query41	52	50	52	50
query42	115	101	98	98
query43	556	507	500	500
query44	1162	824	817	817
query45	179	169	165	165
query46	1161	731	735	731
query47	2048	1916	1937	1916
query48	478	390	392	390
query49	760	389	393	389
query50	877	430	431	430
query51	7377	7259	7125	7125
query52	98	87	87	87
query53	263	176	187	176
query54	548	452	456	452
query55	71	79	79	79
query56	264	241	258	241
query57	1228	1130	1146	1130
query58	217	197	211	197
query59	3331	3015	2960	2960
query60	286	289	252	252
query61	125	106	107	106
query62	772	650	656	650
query63	219	185	190	185
query64	1415	667	636	636
query65	3263	3194	3199	3194
query66	718	308	301	301
query67	15968	15539	15629	15539
query68	3941	599	580	580
query69	478	268	262	262
query70	1220	1089	1121	1089
query71	352	252	254	252
query72	6378	4099	3940	3940
query73	744	351	359	351
query74	10074	8963	9043	8963
query75	3275	2629	2659	2629
query76	1915	1058	1096	1058
query77	479	264	272	264
query78	10526	9673	9591	9591
query79	2130	611	584	584
query80	1388	432	420	420
query81	546	245	242	242
query82	1295	111	113	111
query83	265	145	141	141
query84	289	76	80	76
query85	1025	300	303	300
query86	423	305	281	281
query87	4549	4273	4346	4273
query88	3929	2402	2383	2383
query89	416	287	290	287
query90	1957	185	185	185
query91	188	148	148	148
query92	66	52	51	51
query93	2911	562	560	560
query94	870	297	284	284
query95	368	257	250	250
query96	626	283	289	283
query97	3338	3201	3154	3154
query98	213	208	195	195
query99	1560	1315	1284	1284
Total cold run time: 320845 ms
Total hot run time: 197247 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.64 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c4aa22f0ce11c33d48ba76811d4e299c79eff994, data reload: false

query1	0.03	0.03	0.04
query2	0.06	0.04	0.03
query3	0.23	0.06	0.06
query4	1.63	0.10	0.10
query5	0.51	0.52	0.50
query6	1.14	0.73	0.72
query7	0.02	0.01	0.01
query8	0.04	0.03	0.05
query9	0.56	0.52	0.50
query10	0.56	0.54	0.56
query11	0.15	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.59	0.60
query14	2.91	2.93	2.95
query15	0.90	0.82	0.82
query16	0.37	0.37	0.37
query17	1.05	1.03	1.00
query18	0.24	0.23	0.22
query19	1.87	1.86	2.07
query20	0.01	0.01	0.01
query21	15.36	0.59	0.59
query22	2.90	2.20	1.66
query23	16.94	0.97	0.82
query24	3.24	0.97	0.86
query25	0.16	0.05	0.07
query26	0.53	0.14	0.14
query27	0.05	0.05	0.04
query28	10.75	1.12	1.08
query29	12.59	3.24	3.22
query30	0.24	0.05	0.06
query31	2.85	0.38	0.39
query32	3.24	0.46	0.47
query33	3.04	3.04	3.06
query34	16.91	4.49	4.47
query35	4.53	4.49	4.45
query36	0.66	0.50	0.52
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.18	0.12	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.52 s
Total hot run time: 32.64 s

@w41ter w41ter merged commit 2648152 into branch-3.0 Jan 13, 2025
21 of 22 checks passed
@w41ter w41ter deleted the auto-pick-46770-branch-3.0 branch January 13, 2025 02:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@hello-stephen @doris-robot @w41ter @dataroaring