This repository has been archived by the owner on Dec 13, 2023. It is now read-only.
WAIT Task inside DO_WHILE causing infinite task creation which are already completed #3876
Open
Description
Describe the bug
While running the below workflow it goes into infinite loop
Details
Conductor version: 3.15.0
Persistence implementation: Postgres and MySQL
Queue implementation: MySQL and Postgres
Lock: Redis
Workflow definition:
{
"createTime": 1701489520469,
"createdBy": "owner@email.com",
"updatedBy": "owner@email.com",
"accessPolicy": {},
"name": "test_do_while",
"description": "Workflow details",
"version": 1,
"tasks": [
{
"name": "default__do_while",
"taskReferenceName": "task_1__loop_databricks",
"inputParameters": {},
"type": "DO_WHILE",
"startDelay": 0,
"optional": false,
"asyncComplete": false,
"loopCondition": "if ($.task_1__loop_databricks['iteration'] < 200) { true; } else { false; }",
"loopOver": [
{
"name": "default__sleep",
"taskReferenceName": "task_1__wait_databricks",
"inputParameters": {
"duration": "20 seconds",
"tenantId": "csit"
},
"type": "WAIT",
"startDelay": 0,
"optional": false,
"asyncComplete": false
}
]
}
],
"inputParameters": [],
"outputParameters": {},
"schemaVersion": 2,
"restartable": true,
"workflowStatusListenerEnabled": false,
"ownerEmail": "owner@email.com",
"timeoutPolicy": "ALERT_ONLY",
"timeoutSeconds": 0,
"variables": {},
"inputTemplate": {}
}
Error in conductor server
conductor-server | 2023-12-02 04:56:48.224 ERROR 13 --- [m-task-worker-8] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: 94c1d30a-aef6-4861-be18-4fbfcd03743c could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:48.227 ERROR 13 --- [-task-worker-11] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: 8c683b28-0c10-42dd-894b-2aebead3e3e8 could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:48.234 ERROR 13 --- [m-task-worker-9] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: 976eb165-97af-4451-800b-b506341bd938 could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:48.250 ERROR 13 --- [-task-worker-10] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: 86de75ad-195b-4d18-86f6-7b1280702751 could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:48.293 ERROR 13 --- [-task-worker-12] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: 822de4fa-2291-4516-82b9-bd6ef7f8b0ac could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:48.409 ERROR 13 --- [-task-worker-13] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: 7ed85099-e58f-45e1-845a-c44e141113e5 could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:48.431 ERROR 13 --- [-task-worker-14] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: 784e801d-fcb6-488a-b575-6d476c86a6aa could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:48.466 ERROR 13 --- [-task-worker-15] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: 75e2b4ac-225e-447b-b8ff-b9ab5164c642 could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:48.744 ERROR 13 --- [-task-worker-16] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: a2a4b709-7c51-4b24-a26f-f49cffbcf877 could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:48.880 ERROR 13 --- [-task-worker-17] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: 73526e0e-0ede-4bda-8e4a-9d77d250e947 could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:49.103 ERROR 13 --- [-task-worker-18] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: a47dded2-3d7c-4d25-82eb-021cdd19f288 could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:49.120 ERROR 13 --- [-task-worker-20] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: a725f7d5-da1c-4d42-a453-0550592f8b06 could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:49.121 ERROR 13 --- [-task-worker-21] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: ab0319c0-cb0f-49f5-8a7e-99c04fef1809 could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:49.123 ERROR 13 --- [-task-worker-22] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: abdb849e-6fd9-43cd-b924-5415a933e6bb could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:49.189 ERROR 13 --- [-task-worker-23] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: b2a69eb2-bea1-4764-b986-ad75bb82e9dc could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:49.191 ERROR 13 --- [-task-worker-24] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: c9410bb4-d96e-4ca9-a6be-bd45e6f0ea53 could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:49.192 ERROR 13 --- [m-task-worker-1] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: b3bfbe2f-9c32-41aa-89aa-08bed04c47ce could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:49.194 ERROR 13 --- [m-task-worker-2] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: b4b10f44-1530-4a93-971d-6025762d837b could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:49.196 ERROR 13 --- [m-task-worker-3] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: c5496ec6-2730-43eb-a265-86b06ae35807 could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:49.197 ERROR 13 --- [-task-worker-23] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: c3369393-abb7-4c1f-907d-4d02eda5e9a4 could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:49.207 ERROR 13 --- [m-task-worker-5] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: c26ffe62-4c7f-4d64-a1d5-ef203afc4272 could not be found while executing WAIT
conductor-server | 2023-12-02 04:56:49.210 ERROR 13 --- [m-task-worker-6] c.n.c.c.e.AsyncSystemTaskExecutor : TaskId: b6b9aef8-e941-48e2-a6
To Reproduce
Just goto UI http://localhost:5000
Create the above task definition
Goto workbench
Just trigger this workflow
WARNING - This creates an Infinite loop situation only use this with local conductor setup which can be deleted
Expected behavior
Loop runs and waits for 20 seconds between loop
Screenshots
The workflow is stuck not moving forward.
Additional context
Add any other context about the problem here.