Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fix](group commit) Fix multiple cluster group commit BE select strategy #38644

Merged
merged 1 commit into from
Aug 4, 2024
Merged

[Fix](group commit) Fix multiple cluster group commit BE select strategy #38644

merged 1 commit into from
Aug 4, 2024

Conversation

Yukang-Lian
Copy link
Collaborator

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Yukang-Lian
Copy link
Collaborator Author

run buildall

@github-actions github-actions bot added the doing label Jul 31, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 41419 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 091c7c9247db709b9291a139d40fe6381b845596, data reload: false

------ Round 1 ----------------------------------
q1	17707	4110	4062	4062
q2	2019	205	205	205
q3	10455	1296	1329	1296
q4	10163	840	928	840
q5	7639	2922	2944	2922
q6	219	141	140	140
q7	1023	613	632	613
q8	9444	1733	1925	1733
q9	8747	6579	6589	6579
q10	8788	3844	3852	3844
q11	432	253	261	253
q12	431	235	235	235
q13	17771	2931	2939	2931
q14	263	250	243	243
q15	523	490	482	482
q16	512	385	380	380
q17	953	918	902	902
q18	7955	7298	7252	7252
q19	1635	1220	1204	1204
q20	563	331	344	331
q21	5225	4693	4778	4693
q22	352	292	279	279
Total cold run time: 112819 ms
Total hot run time: 41419 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4110	4037	4037	4037
q2	331	235	221	221
q3	2991	2969	3129	2969
q4	1960	1995	1947	1947
q5	5560	5512	5387	5387
q6	219	134	132	132
q7	2127	1754	1816	1754
q8	3302	3383	3322	3322
q9	8685	8633	8685	8633
q10	3941	4024	3994	3994
q11	541	458	452	452
q12	749	597	630	597
q13	16351	3113	3115	3113
q14	310	270	280	270
q15	533	503	490	490
q16	494	419	405	405
q17	1763	1694	1697	1694
q18	8161	7704	7560	7560
q19	1742	1723	1717	1717
q20	2030	1866	1868	1866
q21	5717	5422	5322	5322
q22	497	465	462	462
Total cold run time: 72114 ms
Total hot run time: 56344 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 170150 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 091c7c9247db709b9291a139d40fe6381b845596, data reload: false

query1	912	372	364	364
query2	6483	1703	1673	1673
query3	6666	212	226	212
query4	20050	17429	17174	17174
query5	3683	515	550	515
query6	287	182	166	166
query7	4599	307	305	305
query8	253	206	193	193
query9	8513	2398	2387	2387
query10	421	279	264	264
query11	10261	9951	10060	9951
query12	123	94	83	83
query13	1625	388	379	379
query14	9552	7764	8342	7764
query15	208	162	160	160
query16	6817	470	465	465
query17	949	570	578	570
query18	1888	277	278	277
query19	192	147	148	147
query20	97	85	84	84
query21	202	100	99	99
query22	4151	3902	3980	3902
query23	33728	33530	33493	33493
query24	10190	3112	3066	3066
query25	669	384	408	384
query26	1844	152	159	152
query27	2974	282	286	282
query28	7527	2058	2028	2028
query29	1236	450	451	450
query30	238	156	155	155
query31	948	795	785	785
query32	106	55	58	55
query33	682	339	316	316
query34	925	491	522	491
query35	873	815	774	774
query36	1042	910	888	888
query37	225	87	90	87
query38	3003	2826	2792	2792
query39	874	802	828	802
query40	263	116	113	113
query41	50	46	45	45
query42	130	107	103	103
query43	451	430	449	430
query44	1150	746	743	743
query45	210	182	180	180
query46	1109	817	782	782
query47	1789	1706	1710	1706
query48	366	290	298	290
query49	886	431	416	416
query50	910	433	441	433
query51	6747	6617	6642	6617
query52	99	90	92	90
query53	252	178	179	178
query54	626	451	454	451
query55	76	78	76	76
query56	291	254	246	246
query57	1125	1037	1046	1037
query58	277	276	278	276
query59	2577	2339	2391	2339
query60	283	270	274	270
query61	101	96	127	96
query62	891	675	668	668
query63	222	189	190	189
query64	5707	1903	1870	1870
query65	3143	3096	3093	3093
query66	1053	333	353	333
query67	15170	14706	14830	14706
query68	4456	560	575	560
query69	715	367	322	322
query70	1145	1058	1062	1058
query71	394	283	281	281
query72	7675	2706	2505	2505
query73	792	326	328	326
query74	6015	5678	5627	5627
query75	3336	2713	2744	2713
query76	2791	1396	1419	1396
query77	574	315	311	311
query78	9409	8857	8895	8857
query79	1877	552	527	527
query80	989	531	502	502
query81	550	222	227	222
query82	851	130	128	128
query83	254	167	171	167
query84	259	78	77	77
query85	1158	302	303	302
query86	444	310	294	294
query87	3253	3149	3074	3074
query88	2887	2409	2411	2409
query89	391	290	292	290
query90	1633	198	200	198
query91	124	101	161	101
query92	59	48	48	48
query93	1862	618	625	618
query94	793	305	273	273
query95	394	271	271	271
query96	606	295	291	291
query97	3212	3023	3043	3023
query98	223	203	196	196
query99	1632	1294	1273	1273
Total cold run time: 261723 ms
Total hot run time: 170150 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.02 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 091c7c9247db709b9291a139d40fe6381b845596, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.07	0.07
query5	0.48	0.48	0.48
query6	1.15	0.70	0.72
query7	0.02	0.01	0.01
query8	0.06	0.05	0.04
query9	0.58	0.51	0.51
query10	0.56	0.56	0.58
query11	0.16	0.11	0.11
query12	0.15	0.13	0.13
query13	0.61	0.60	0.60
query14	0.77	0.80	0.77
query15	0.88	0.87	0.86
query16	0.34	0.35	0.35
query17	0.99	1.00	1.03
query18	0.22	0.20	0.21
query19	1.81	1.73	1.75
query20	0.01	0.01	0.01
query21	15.41	0.80	0.68
query22	3.97	8.03	1.31
query23	17.73	1.27	1.24
query24	2.28	0.22	0.23
query25	0.19	0.09	0.09
query26	0.32	0.22	0.22
query27	0.46	0.24	0.23
query28	13.16	1.02	0.98
query29	12.51	3.36	3.37
query30	0.26	0.07	0.05
query31	2.85	0.40	0.41
query32	3.24	0.50	0.47
query33	2.95	2.93	2.99
query34	15.45	4.25	4.23
query35	4.30	4.28	4.29
query36	0.68	0.48	0.48
query37	0.20	0.16	0.17
query38	0.17	0.16	0.15
query39	0.04	0.04	0.03
query40	0.15	0.13	0.14
query41	0.10	0.05	0.05
query42	0.05	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 107.32 s
Total hot run time: 30.02 s

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 2, 2024

PR approved by anyone and no changes requested.

dataroaring
dataroaring approved these changes Aug 4, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 4, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 4, 2024
@dataroaring dataroaring merged commit ef4fba6 into apache:master Aug 4, 2024
29 of 31 checks passed
Yukang-Lian added a commit to Yukang-Lian/doris that referenced this pull request Aug 7, 2024
@Yukang-Lian
[Fix](group commit) Fix multiple cluster group commit BE select strat…
27f3671 
…egy (apache#38644)
@Yukang-Lian Yukang-Lian mentioned this pull request Aug 7, 2024
Merged
Yukang-Lian added a commit to Yukang-Lian/doris that referenced this pull request Aug 7, 2024
@Yukang-Lian
[Fix](group commit) Fix multiple cluster group commit BE select strat…
9e18433 
…egy (apache#38644)
yiguolei pushed a commit that referenced this pull request Aug 7, 2024
@Yukang-Lian
[cherry-pick](branch-2.1) Pick "[Fix](group commit) Fix multiple clus…
e083dc2 
…ter group commit BE select strategy (#38644)" (#39010)

## Proposed changes

Pick #38644 

<!--Describe your changes.-->
dataroaring pushed a commit that referenced this pull request Aug 11, 2024
@Yukang-Lian @dataroaring
[Fix](group commit) Fix multiple cluster group commit BE select strat…
62bc25c 
…egy (#38644)
dataroaring pushed a commit that referenced this pull request Aug 16, 2024
@Yukang-Lian @dataroaring
[Fix](group commit) Fix multiple cluster group commit BE select strat…
2c284b8 
…egy (#38644)
@gavinchou gavinchou mentioned this pull request Oct 13, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dataroaring dataroaring dataroaring approved these changes

@mymeiyi mymeiyi mymeiyi approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.2-merged doing reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@Yukang-Lian @doris-robot @mymeiyi @dataroaring @yiguolei