Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

branch-3.0: [fix](hive) Incorrect location conversion #46362 #46515

Merged
merged 1 commit into from
Jan 10, 2025
Merged

branch-3.0: [fix](hive) Incorrect location conversion #46362 #46515

merged 1 commit into from
Jan 10, 2025

Conversation

github-actions[bot]
Copy link
Contributor

@github-actions github-actions bot commented Jan 7, 2025

Cherry-picked from #46362

@wuwenchi
[fix](hive) Incorrect location conversion (#46362)
390994f 
### What problem does this PR solve?

Problem Summary:

For the "original_write_path" field, when the location is of the S3
type, it should not be converted into the S3 schema. Instead, the
original location should be retained.
@Thearas
Copy link
Contributor

Thearas commented Jan 7, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Jan 7, 2025
@dataroaring dataroaring reopened this Jan 7, 2025
@Thearas
Copy link
Contributor

Thearas commented Jan 7, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40783 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 390994f46475e857d63a8cf5ddc8b74c56072414, data reload: false

------ Round 1 ----------------------------------
q1	17572	7411	7231	7231
q2	2050	176	167	167
q3	10604	1104	1161	1104
q4	10519	726	689	689
q5	7786	2821	2852	2821
q6	244	146	143	143
q7	975	621	611	611
q8	9369	1979	1987	1979
q9	6729	6438	6405	6405
q10	7049	2292	2301	2292
q11	465	264	253	253
q12	403	205	209	205
q13	17766	2971	2988	2971
q14	241	213	207	207
q15	587	525	529	525
q16	695	616	600	600
q17	966	527	532	527
q18	7230	6842	6719	6719
q19	1395	1058	938	938
q20	485	206	193	193
q21	4135	3204	3252	3204
q22	1070	1002	999	999
Total cold run time: 108335 ms
Total hot run time: 40783 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7270	7183	7165	7165
q2	330	242	243	242
q3	2898	2921	2914	2914
q4	2027	1826	1855	1826
q5	5707	5715	5728	5715
q6	225	142	138	138
q7	2285	1802	1836	1802
q8	3370	3525	3527	3525
q9	8825	8892	8904	8892
q10	3569	3519	3519	3519
q11	614	494	502	494
q12	821	620	588	588
q13	8540	3136	3137	3136
q14	305	279	273	273
q15	587	524	523	523
q16	726	674	659	659
q17	1827	1642	1599	1599
q18	8191	7719	7742	7719
q19	1679	1623	1522	1522
q20	2117	1858	1873	1858
q21	5619	5483	5307	5307
q22	1148	1024	1023	1023
Total cold run time: 68680 ms
Total hot run time: 60439 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197917 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 390994f46475e857d63a8cf5ddc8b74c56072414, data reload: false

query1	1309	926	921	921
query2	6220	2048	2024	2024
query3	10795	4295	4394	4295
query4	66867	29460	23618	23618
query5	4949	450	452	450
query6	407	177	168	168
query7	5586	315	308	308
query8	312	232	239	232
query9	8938	2727	2705	2705
query10	462	270	250	250
query11	17218	15184	15700	15184
query12	152	115	104	104
query13	1568	475	440	440
query14	10260	7688	7332	7332
query15	211	174	182	174
query16	7201	500	489	489
query17	1088	610	606	606
query18	1896	321	334	321
query19	228	157	166	157
query20	128	114	112	112
query21	217	105	108	105
query22	4842	4427	4530	4427
query23	34955	34077	34043	34043
query24	6185	3003	2913	2913
query25	554	424	431	424
query26	655	171	174	171
query27	1767	364	357	357
query28	4093	2478	2463	2463
query29	709	498	471	471
query30	245	179	180	179
query31	1027	850	851	850
query32	64	59	55	55
query33	456	290	287	287
query34	921	537	524	524
query35	855	737	724	724
query36	1136	969	989	969
query37	123	79	76	76
query38	4106	4116	3979	3979
query39	1524	1447	1527	1447
query40	210	101	94	94
query41	47	48	46	46
query42	116	106	104	104
query43	531	498	502	498
query44	1205	831	830	830
query45	185	167	171	167
query46	1178	744	743	743
query47	1980	1901	1916	1901
query48	474	388	383	383
query49	719	391	374	374
query50	842	437	432	432
query51	7433	7221	7323	7221
query52	99	88	89	88
query53	253	184	193	184
query54	563	450	459	450
query55	76	75	79	75
query56	263	244	246	244
query57	1229	1121	1089	1089
query58	220	230	223	223
query59	3032	3011	3021	3011
query60	264	252	253	252
query61	112	105	107	105
query62	780	669	652	652
query63	228	195	183	183
query64	1367	677	636	636
query65	3273	3251	3188	3188
query66	622	302	305	302
query67	16068	15856	15502	15502
query68	4134	582	566	566
query69	441	268	264	264
query70	1183	1131	1091	1091
query71	338	249	253	249
query72	6418	4088	3963	3963
query73	762	352	355	352
query74	10260	8998	8986	8986
query75	3384	2697	2655	2655
query76	1850	1052	1127	1052
query77	503	263	264	263
query78	10671	9741	9596	9596
query79	1122	599	618	599
query80	844	423	424	423
query81	503	245	239	239
query82	1306	120	119	119
query83	166	146	142	142
query84	290	84	81	81
query85	832	296	293	293
query86	319	296	266	266
query87	4559	4302	4228	4228
query88	3452	2437	2391	2391
query89	406	302	289	289
query90	2086	189	190	189
query91	182	146	145	145
query92	58	50	50	50
query93	1078	547	552	547
query94	767	301	298	298
query95	359	256	260	256
query96	622	281	287	281
query97	3350	3205	3199	3199
query98	214	203	199	199
query99	1786	1315	1327	1315
Total cold run time: 317843 ms
Total hot run time: 197917 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.91 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 390994f46475e857d63a8cf5ddc8b74c56072414, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.04	0.03
query3	0.23	0.07	0.06
query4	1.64	0.10	0.10
query5	0.51	0.52	0.52
query6	1.14	0.73	0.72
query7	0.03	0.02	0.02
query8	0.04	0.03	0.03
query9	0.57	0.51	0.53
query10	0.55	0.55	0.55
query11	0.14	0.11	0.11
query12	0.14	0.11	0.11
query13	0.61	0.60	0.60
query14	2.89	2.93	2.91
query15	0.89	0.82	0.83
query16	0.40	0.40	0.37
query17	1.06	0.98	1.00
query18	0.23	0.22	0.22
query19	1.98	1.79	2.00
query20	0.02	0.01	0.01
query21	15.36	0.58	0.58
query22	2.60	2.26	2.18
query23	17.00	0.92	0.90
query24	3.45	0.89	0.52
query25	0.38	0.15	0.08
query26	0.24	0.13	0.14
query27	0.05	0.05	0.04
query28	10.92	1.10	1.07
query29	12.65	3.29	3.30
query30	0.25	0.06	0.06
query31	2.86	0.40	0.38
query32	3.23	0.46	0.47
query33	2.95	3.03	3.03
query34	17.07	4.55	4.48
query35	4.56	4.44	4.53
query36	0.69	0.50	0.48
query37	0.09	0.06	0.06
query38	0.05	0.03	0.03
query39	0.03	0.02	0.03
query40	0.15	0.14	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.04
Total cold run time: 107.91 s
Total hot run time: 32.91 s

@morningman morningman merged commit 65af892 into branch-3.0 Jan 10, 2025
21 of 22 checks passed
@github-actions github-actions bot deleted the auto-pick-46362-branch-3.0 branch January 10, 2025 05:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@Thearas @doris-robot @morningman @dataroaring @wuwenchi