Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](ES catalog)Fix query long value exception with doc_value #46554

Merged
merged 1 commit into from
Jan 8, 2025
Merged

[fix](ES catalog)Fix query long value exception with doc_value #46554

merged 1 commit into from
Jan 8, 2025

Conversation

qidaye
Copy link
Contributor

@qidaye qidaye commented Jan 7, 2025 •
edited
Loading

What problem does this PR solve?

Issue Number: close #46553

Problem Summary:

When parsing number value from pure_doc_value or array, we should get the first value, not the whole array.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Jan 7, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@qidaye
Copy link
Contributor Author

qidaye commented Jan 7, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32784 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d688881b708680bde25df734263b3be5083a9439, data reload: false

------ Round 1 ----------------------------------
q1	17575	6151	6038	6038
q2	2050	312	182	182
q3	10395	1235	748	748
q4	10216	897	445	445
q5	7552	2228	2047	2047
q6	209	181	162	162
q7	921	743	608	608
q8	9235	1376	1189	1189
q9	5331	4932	4904	4904
q10	6769	2315	1863	1863
q11	490	277	262	262
q12	347	356	221	221
q13	17760	3665	3083	3083
q14	234	234	218	218
q15	568	499	486	486
q16	627	620	600	600
q17	588	871	346	346
q18	7186	6479	6357	6357
q19	2800	981	550	550
q20	299	313	183	183
q21	2879	2230	1986	1986
q22	362	337	306	306
Total cold run time: 104393 ms
Total hot run time: 32784 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6362	6250	6247	6247
q2	236	333	232	232
q3	2302	2680	2338	2338
q4	1394	1844	1392	1392
q5	4364	4763	4922	4763
q6	188	174	141	141
q7	2066	1960	1845	1845
q8	2697	2841	2666	2666
q9	7323	7356	7318	7318
q10	3098	3356	2850	2850
q11	573	501	480	480
q12	643	717	593	593
q13	3513	3903	3248	3248
q14	287	315	277	277
q15	576	520	489	489
q16	661	687	645	645
q17	1242	1736	1247	1247
q18	7725	7608	7434	7434
q19	830	1209	1110	1110
q20	2001	1996	1921	1921
q21	5742	5335	4893	4893
q22	628	641	570	570
Total cold run time: 54451 ms
Total hot run time: 52699 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198107 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d688881b708680bde25df734263b3be5083a9439, data reload: false

query1	1317	939	923	923
query2	6088	2455	2297	2297
query3	11120	4975	4998	4975
query4	33484	23713	23597	23597
query5	4005	627	467	467
query6	294	209	209	209
query7	3982	504	312	312
query8	300	256	223	223
query9	9457	2823	2793	2793
query10	464	297	249	249
query11	17810	15389	15032	15032
query12	155	105	101	101
query13	1556	526	417	417
query14	9867	7719	7632	7632
query15	232	212	189	189
query16	7325	623	490	490
query17	1566	783	632	632
query18	1870	418	342	342
query19	238	217	155	155
query20	117	113	110	110
query21	218	131	112	112
query22	4712	4750	4695	4695
query23	34500	33369	33634	33369
query24	6371	2394	2325	2325
query25	451	455	399	399
query26	719	288	151	151
query27	1942	474	340	340
query28	5288	2604	2580	2580
query29	540	558	430	430
query30	211	182	146	146
query31	997	978	878	878
query32	70	63	61	61
query33	473	369	298	298
query34	772	849	522	522
query35	804	845	765	765
query36	1028	1051	961	961
query37	110	101	76	76
query38	4234	4273	4218	4218
query39	1564	1451	1479	1451
query40	201	116	111	111
query41	49	44	45	44
query42	117	106	107	106
query43	523	537	496	496
query44	1343	835	848	835
query45	182	179	171	171
query46	900	1063	668	668
query47	2006	2043	1949	1949
query48	397	421	334	334
query49	711	478	427	427
query50	634	650	405	405
query51	7171	7107	6927	6927
query52	99	98	92	92
query53	228	260	190	190
query54	492	512	425	425
query55	82	84	79	79
query56	252	271	237	237
query57	1269	1257	1204	1204
query58	269	238	248	238
query59	3155	3459	3016	3016
query60	310	266	258	258
query61	114	128	108	108
query62	896	840	770	770
query63	232	188	201	188
query64	2721	1011	681	681
query65	3400	3267	3248	3248
query66	858	411	305	305
query67	16542	15899	15574	15574
query68	8568	745	567	567
query69	479	307	249	249
query70	1222	1134	1133	1133
query71	455	278	246	246
query72	6276	3864	3871	3864
query73	659	759	370	370
query74	10061	9200	8844	8844
query75	4583	3149	2668	2668
query76	4483	1173	771	771
query77	776	352	295	295
query78	10188	10006	9638	9638
query79	3697	804	595	595
query80	729	517	422	422
query81	483	267	300	267
query82	332	162	127	127
query83	193	167	145	145
query84	288	96	72	72
query85	729	380	295	295
query86	347	311	312	311
query87	4874	4459	4501	4459
query88	3226	2266	2236	2236
query89	442	335	296	296
query90	2053	191	187	187
query91	135	133	104	104
query92	69	57	51	51
query93	2106	880	574	574
query94	650	407	289	289
query95	334	266	254	254
query96	491	623	276	276
query97	2935	2953	2860	2860
query98	224	209	195	195
query99	1657	1547	1439	1439
Total cold run time: 295148 ms
Total hot run time: 198107 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.86% (10129/26063)
Line Coverage: 29.91% (85690/286530)
Region Coverage: 29.01% (43725/150735)
Branch Coverage: 25.55% (22318/87360)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d688881b708680bde25df734263b3be5083a9439_d688881b708680bde25df734263b3be5083a9439/report/index.html

@doris-robot
Copy link

ClickBench: Total hot run time: 31.7 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d688881b708680bde25df734263b3be5083a9439, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.04	0.03
query3	0.23	0.07	0.08
query4	1.61	0.10	0.11
query5	0.43	0.42	0.41
query6	1.16	0.66	0.65
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.59	0.51	0.52
query10	0.56	0.55	0.55
query11	0.14	0.12	0.11
query12	0.14	0.11	0.11
query13	0.60	0.61	0.58
query14	2.74	2.76	2.74
query15	0.89	0.82	0.83
query16	0.40	0.38	0.37
query17	1.04	1.04	1.06
query18	0.22	0.20	0.21
query19	1.90	1.84	1.99
query20	0.02	0.00	0.01
query21	15.36	0.93	0.59
query22	0.75	0.84	0.68
query23	15.23	1.37	0.60
query24	2.87	1.92	1.16
query25	0.24	0.17	0.09
query26	0.21	0.16	0.13
query27	0.05	0.05	0.04
query28	14.18	1.48	1.05
query29	12.58	3.99	3.31
query30	0.25	0.09	0.07
query31	2.80	0.58	0.39
query32	3.23	0.54	0.47
query33	3.09	3.06	3.07
query34	16.70	5.13	4.51
query35	4.58	4.58	4.57
query36	0.65	0.51	0.48
query37	0.09	0.06	0.06
query38	0.04	0.03	0.03
query39	0.03	0.02	0.02
query40	0.17	0.14	0.12
query41	0.08	0.02	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.09 s
Total hot run time: 31.7 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 7, 2025
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 7, 2025

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 7, 2025

PR approved by anyone and no changes requested.

eldenmoon
eldenmoon approved these changes Jan 8, 2025
View reviewed changes
Copy link
Member

@eldenmoon eldenmoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qidaye qidaye merged commit ff61365 into apache:master Jan 8, 2025
30 of 32 checks passed
@qidaye qidaye deleted the fix_es_catalog_query_error branch January 8, 2025 02:06
github-actions bot pushed a commit that referenced this pull request Jan 8, 2025
@qidaye
[fix](ES catalog)Fix query long value exception with doc_value (#46554)
c25e447 
Issue Number: close #46553

Problem Summary:

When parsing number value from pure_doc_value or array, we should get
the first value, not the whole array.
@github-actions github-actions bot mentioned this pull request Jan 8, 2025
Merged
github-actions bot pushed a commit that referenced this pull request Jan 8, 2025
@qidaye
[fix](ES catalog)Fix query long value exception with doc_value (#46554)
0e6ec85 
Issue Number: close #46553

Problem Summary:

When parsing number value from pure_doc_value or array, we should get
the first value, not the whole array.
@github-actions github-actions bot mentioned this pull request Jan 8, 2025
Merged
qidaye added a commit that referenced this pull request Jan 8, 2025
@github-actions @qidaye
branch-2.1: [fix](ES catalog)Fix query long value exception with doc_…
4d0037a 
…value #46554 (#46581)

Cherry-picked from #46554

Co-authored-by: qiye <luen@selectdb.com>
qidaye added a commit that referenced this pull request Jan 9, 2025
@github-actions @qidaye
branch-3.0: [fix](ES catalog)Fix query long value exception with doc_…
1d2d031 
…value #46554 (#46580)

Cherry-picked from #46554

Co-authored-by: qiye <luen@selectdb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@eldenmoon eldenmoon eldenmoon approved these changes

@morningman morningman morningman approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.4-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Bug] ES catalog query error
5 participants
@qidaye @Thearas @doris-robot @morningman @eldenmoon