Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](catalog) remove backend in black list from candidate backends for external table (#38984) #39451

Merged
merged 1 commit into from
Aug 19, 2024
Merged

[fix](catalog) remove backend in black list from candidate backends for external table (#38984) #39451

merged 1 commit into from
Aug 19, 2024

Conversation

morningman
Copy link
Contributor

When select backends for external table's query,
skip the backends in black list

@morningman
[fix](catalog) remove backend in black list from candidate backends f…
dffb60d 
…or external table (apache#38984)

When select backends for external table's query,
skip the backends in black list
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morningman morningman mentioned this pull request Aug 15, 2024
Merged
@morningman
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38782 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dffb60d2511c48c344d955a8c78b6e49c7664518, data reload: false

------ Round 1 ----------------------------------
q1	18137	4456	4341	4341
q2	2049	229	214	214
q3	11820	1069	1112	1069
q4	11043	762	836	762
q5	7826	2878	2830	2830
q6	266	154	154	154
q7	1004	648	647	647
q8	9634	2091	2089	2089
q9	7530	6626	6634	6626
q10	7119	2259	2288	2259
q11	520	291	280	280
q12	440	277	268	268
q13	18821	3203	3279	3203
q14	334	290	279	279
q15	593	581	557	557
q16	888	431	431	431
q17	1007	702	767	702
q18	8225	6840	6847	6840
q19	6192	1175	1072	1072
q20	1210	369	368	368
q21	3897	2976	2769	2769
q22	1141	1024	1022	1022
Total cold run time: 119696 ms
Total hot run time: 38782 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4985	4314	4313	4313
q2	505	301	309	301
q3	2846	2795	2733	2733
q4	1943	1734	1706	1706
q5	5714	5665	5583	5583
q6	231	150	144	144
q7	2079	1756	1824	1756
q8	3320	3460	3494	3460
q9	8853	8758	8892	8758
q10	3560	3320	3272	3272
q11	646	530	534	530
q12	838	655	644	644
q13	15782	3043	3163	3043
q14	337	304	322	304
q15	569	531	526	526
q16	503	474	483	474
q17	1817	1560	1535	1535
q18	8178	7778	7668	7668
q19	9867	1598	1653	1598
q20	2140	1894	1893	1893
q21	5627	5290	5462	5290
q22	1217	1070	1077	1070
Total cold run time: 81557 ms
Total hot run time: 56601 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196789 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dffb60d2511c48c344d955a8c78b6e49c7664518, data reload: false

query1	1322	924	885	885
query2	6718	1931	1909	1909
query3	10619	4000	3769	3769
query4	55548	24451	23408	23408
query5	5971	638	648	638
query6	530	189	186	186
query7	6423	312	309	309
query8	464	367	370	367
query9	9066	2494	2464	2464
query10	553	331	336	331
query11	18053	15058	15218	15058
query12	185	125	128	125
query13	1663	429	419	419
query14	11929	7522	7788	7522
query15	291	201	200	200
query16	7641	557	511	511
query17	1188	656	606	606
query18	1980	339	340	339
query19	299	179	185	179
query20	158	146	146	146
query21	260	146	146	146
query22	4496	4448	4615	4448
query23	35440	34083	34192	34083
query24	5805	3054	3032	3032
query25	567	434	439	434
query26	704	187	186	186
query27	1788	302	300	300
query28	4126	2208	2184	2184
query29	722	477	448	448
query30	239	188	184	184
query31	1065	836	831	831
query32	97	76	79	76
query33	538	332	345	332
query34	959	496	499	496
query35	892	778	760	760
query36	1151	963	957	957
query37	165	105	101	101
query38	3977	3901	3921	3901
query39	1513	1479	1484	1479
query40	239	159	156	156
query41	140	138	138	138
query42	147	116	119	116
query43	533	518	510	510
query44	1166	792	789	789
query45	227	190	192	190
query46	1197	790	829	790
query47	1915	1822	1808	1808
query48	416	343	339	339
query49	931	614	583	583
query50	931	472	471	471
query51	6844	6707	6801	6707
query52	119	112	110	110
query53	311	233	230	230
query54	633	508	504	504
query55	90	89	88	88
query56	334	309	309	309
query57	1227	1122	1089	1089
query58	299	303	319	303
query59	3028	2920	2839	2839
query60	359	334	331	331
query61	148	148	146	146
query62	812	695	672	672
query63	264	231	228	228
query64	3341	1898	1858	1858
query65	3277	3184	3175	3175
query66	1041	671	660	660
query67	15389	14922	14868	14868
query68	6787	588	594	588
query69	725	410	332	332
query70	1208	1103	1198	1103
query71	541	319	323	319
query72	6507	2385	2058	2058
query73	883	359	363	359
query74	9161	8736	8942	8736
query75	3766	2807	2784	2784
query76	4102	1127	1086	1086
query77	904	455	448	448
query78	9930	9876	9143	9143
query79	7850	568	577	568
query80	1470	625	639	625
query81	573	260	266	260
query82	676	162	162	162
query83	349	214	214	214
query84	345	98	100	98
query85	959	363	355	355
query86	416	299	331	299
query87	4538	4162	4267	4162
query88	4405	2543	2532	2532
query89	472	326	321	321
query90	1964	234	235	234
query91	158	128	128	128
query92	91	78	75	75
query93	4365	563	554	554
query94	760	328	323	323
query95	396	304	298	298
query96	680	287	291	287
query97	3316	3043	3092	3043
query98	246	225	222	222
query99	1749	1322	1317	1317
Total cold run time: 330755 ms
Total hot run time: 196789 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.69 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dffb60d2511c48c344d955a8c78b6e49c7664518, data reload: false

query1	0.05	0.04	0.04
query2	0.09	0.05	0.04
query3	0.23	0.05	0.06
query4	1.66	0.09	0.08
query5	0.48	0.47	0.50
query6	1.13	0.72	0.72
query7	0.02	0.02	0.02
query8	0.05	0.05	0.05
query9	0.55	0.50	0.49
query10	0.58	0.55	0.56
query11	0.15	0.13	0.12
query12	0.15	0.13	0.14
query13	0.61	0.63	0.60
query14	0.76	0.79	0.78
query15	0.87	0.84	0.82
query16	0.39	0.36	0.38
query17	1.02	1.02	1.02
query18	0.24	0.23	0.23
query19	1.86	1.69	1.70
query20	0.01	0.01	0.02
query21	15.40	0.84	0.64
query22	4.92	6.63	2.54
query23	18.30	1.31	1.22
query24	2.16	0.21	0.22
query25	0.15	0.09	0.09
query26	0.30	0.21	0.21
query27	0.46	0.24	0.23
query28	13.21	1.03	1.01
query29	12.63	3.30	3.30
query30	0.38	0.20	0.19
query31	2.80	0.39	0.40
query32	3.25	0.50	0.49
query33	2.95	2.99	2.97
query34	17.04	4.32	4.37
query35	4.54	4.40	4.39
query36	0.67	0.50	0.49
query37	0.20	0.17	0.17
query38	0.16	0.16	0.17
query39	0.06	0.06	0.07
query40	0.18	0.16	0.14
query41	0.12	0.07	0.07
query42	0.08	0.07	0.08
query43	0.07	0.06	0.07
Total cold run time: 110.93 s
Total hot run time: 31.69 s

dataroaring
dataroaring approved these changes Aug 16, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 16, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

kaka11chen
kaka11chen approved these changes Aug 19, 2024
View reviewed changes
Copy link
Contributor

@kaka11chen kaka11chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morningman morningman merged commit f0a2ce5 into apache:master Aug 19, 2024
29 of 31 checks passed
dataroaring pushed a commit that referenced this pull request Aug 26, 2024
@morningman @dataroaring
[fix](catalog) remove backend in black list from candidate backends f…
45f6a51 
…or external table (#38984) (#39451)

When select backends for external table's query,
skip the backends in black list
@gavinchou gavinchou mentioned this pull request Oct 13, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dataroaring dataroaring dataroaring approved these changes

@kaka11chen kaka11chen kaka11chen approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.2-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@morningman @doris-robot @kaka11chen @dataroaring