Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Improve]For different query, merge has different meanings, so we need to split to MERGE and MERGE_AND_ORDER #1139

Closed
yanhuqing666 opened this issue Apr 23, 2019 · 1 comment
Closed

[Improve]For different query, merge has different meanings, so we need to split to MERGE and MERGE_AND_ORDER #1139

yanhuqing666 opened this issue Apr 23, 2019 · 1 comment
Assignees
@FlyingMao
Labels
Explain Improvement resolve problem has been fixed by developer
Milestone
2.19.05.0

Comments

@yanhuqing666
Copy link
Member

We split old MERGE to new MERGE AND MERGE_AND_ORDER
New MERGE means just send all rowsResponse from differnt date node to next step to handle.

for exmaple:

mysql> explain select count(*) from sharding_two_node;
+-----------------+---------------+--------------------------------------------------------------------------+
| DATA_NODE       | TYPE          | SQL/REF                                                                  |
+-----------------+---------------+--------------------------------------------------------------------------+
| dn1_0           | BASE SQL      | select COUNT(*) as `_$COUNT$_rpda_0` from  `sharding_two_node` LIMIT 100 |
| dn2_0           | BASE SQL      | select COUNT(*) as `_$COUNT$_rpda_0` from  `sharding_two_node` LIMIT 100 |
| merge_1         | MERGE         | dn1_0; dn2_0                                                             |
| aggregate_1     | AGGREGATE     | merge_1                                                                  |
| limit_1         | LIMIT         | aggregate_1                                                              |
| shuffle_field_1 | SHUFFLE_FIELD | limit_1                                                                  |
+-----------------+---------------+--------------------------------------------------------------------------+
6 rows in set (1.32 sec)

MERGE_AND_ORDER mean order all rowsResponse from differnt date node and used them to join in dble or return to client.
for exmaple:

mysql> explain select a.id as c,* from sharding_two_node a inner join sharding_four_node b on a.id =b.id ;
+-------------------+-----------------+--------------------------------------------------------------------------------+
| DATA_NODE         | TYPE            | SQL/REF                                                                        |
+-------------------+-----------------+--------------------------------------------------------------------------------+
| dn1_0             | BASE SQL        | select `a`.`id`,`a`.`name` from  `sharding_two_node` `a` ORDER BY `a`.`id` ASC |
| dn2_0             | BASE SQL        | select `a`.`id`,`a`.`name` from  `sharding_two_node` `a` ORDER BY `a`.`id` ASC |
| merge_and_order_1 | MERGE_AND_ORDER | dn1_0; dn2_0                                                                   |
| shuffle_field_1   | SHUFFLE_FIELD   | merge_and_order_1                                                              |
| dn1_1             | BASE SQL        | select `b`.`id`,`b`.`c` from  `sharding_four_node` `b` ORDER BY `b`.`id` ASC   |
| dn2_1             | BASE SQL        | select `b`.`id`,`b`.`c` from  `sharding_four_node` `b` ORDER BY `b`.`id` ASC   |
| dn3_0             | BASE SQL        | select `b`.`id`,`b`.`c` from  `sharding_four_node` `b` ORDER BY `b`.`id` ASC   |
| dn4_0             | BASE SQL        | select `b`.`id`,`b`.`c` from  `sharding_four_node` `b` ORDER BY `b`.`id` ASC   |
| merge_and_order_2 | MERGE_AND_ORDER | dn1_1; dn2_1; dn3_0; dn4_0                                                     |
| shuffle_field_3   | SHUFFLE_FIELD   | merge_and_order_2                                                              |
| join_1            | JOIN            | shuffle_field_1; shuffle_field_3                                               |
| shuffle_field_2   | SHUFFLE_FIELD   | join_1                                                                         |
+-------------------+-----------------+--------------------------------------------------------------------------------+
12 rows in set (0.01 sec)
@yanhuqing666 yanhuqing666 added this to the 2.19.05.0 milestone Apr 23, 2019
@yanhuqing666 yanhuqing666 self-assigned this Apr 23, 2019
yanhuqing666 added a commit that referenced this issue Apr 23, 2019
@yanhuqing666
split merge hander #1139
5444284
@yanhuqing666 yanhuqing666 mentioned this issue Apr 23, 2019
Merged
yanhuqing666 added a commit that referenced this issue Apr 26, 2019
@yanhuqing666
split merge hander #1139 (#1140)
9e7e94c
@yanhuqing666 yanhuqing666 added the resolve problem has been fixed by developer label Apr 26, 2019
yanhuqing666 added a commit that referenced this issue Apr 28, 2019
@yanhuqing666
reachedConCount thread safe #1139
6398b0b
@yanhuqing666 yanhuqing666 mentioned this issue Apr 28, 2019
Closed
@FlyingMao FlyingMao assigned FlyingMao and unassigned irene-coming Jun 11, 2019
@FlyingMao
Copy link

verified version： dble-9.9.9.9-42aeaab7fc3d719a8e918dce18176909ed35b340-20190610120741

Lordess pushed a commit to Lordess/dble that referenced this issue May 28, 2020
@yanhuqing666 @Lordess
split merge hander actiontech#1139 (actiontech#1140)
17b0580
Lordess pushed a commit to Lordess/dble that referenced this issue May 28, 2020
@yanhuqing666 @Lordess
reachedConCount thread safe actiontech#1139
ca67bc2
@Lordess Lordess mentioned this issue May 28, 2020
Merged
yanhuqing666 added a commit that referenced this issue May 29, 2020
@Lordess @yanhuqing666
release 2.19.03.4/lts (#1865)
2173c4e 
* split merge hander #1139 (#1140)

* reachedConCount thread safe #1139

* import feature from 2.19.11.0/rel to 2.19.03/lts

 * #1564 [Improve] add STAGE for show @@connection.sql

* import feature from 2.19.11.0/rel to 2.19.03/lts

* #1575 [Improve]add an manager command to show recently(doing or done) query detail stage

* release 2.19.03.4/lts

Co-authored-by: tiger.yan <yanhuqing666@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@FlyingMao FlyingMao

Labels
Explain Improvement resolve problem has been fixed by developer
Projects
None yet
Milestone
2.19.05.0
Development

No branches or pull requests
3 participants
@irene-coming @yanhuqing666 @FlyingMao