We have lost the previous tasks in the WebServer #31179
Description
Apache Airflow version
2.6.0
What happened
After upgrading Airflow to 2.6.0 from 2.5.1
We are losing the previous tasks on the Airflow web server GRID . These tasks are executed correctly however they are not displayed.
The behavior is curious, since the previous tasks are finished. An example is that if I mark the last one as Failed. The Grid is updated and shows me the tasks again.
Also if I select this option. You can see that the tasks are found, but they are never shown.
But finally I do not do a clear or anything, mysteriously the tasks in the Grid are loaded correctly (sometimes are loaded with the correct state "sucessfull" other time as in the image without any state)
As you can see in the image, all the task are 'found' but not loaded on the grid. Even all the previous task are finished succesfully.
Also if I check LOGS for previous NONE status task, logs are shown.
We are also seeing other behavior. Since we 'lose' the previous tasks, the DagRuns never finish and this results in the next DagRuns never being executed. That is, our Dummy task (Start) is not launched because since we do not have the previous tasks, we do not have their states.
Other example.
This is the GRID from a DAG.
Also If I check for previous dags, this is what is shown.
If I mark as success all 'start' task, suddenly GRID shows previous task, in this case with it's correct state.
What you think should happen instead
All the task should be displayed correctly with their correct state
How to reproduce
We don't know, we have days where all the DagRuns are with their correct tasks and status. Other cases where having the DAG correctly starts to go wrong and we end up losing the tasks. And other cases where nothing is affected.
Operating System
"Debian GNU/Linux 11 (bullseye)" NAME="Debian GNU/Linux"
Versions of Apache Airflow Providers
apache-airflow-providers-amazon==8.0.0
apache-airflow-providers-celery==3.1.0
apache-airflow-providers-cncf-kubernetes==6.1.0
apache-airflow-providers-common-sql==1.4.0
apache-airflow-providers-docker==3.6.0
apache-airflow-providers-elasticsearch==4.4.0
apache-airflow-providers-ftp==3.3.1
apache-airflow-providers-google==10.0.0
apache-airflow-providers-grpc==3.1.0
apache-airflow-providers-hashicorp==3.3.1
apache-airflow-providers-http==4.3.0
apache-airflow-providers-imap==3.1.1
apache-airflow-providers-microsoft-azure==6.0.0
apache-airflow-providers-mysql==5.0.0
apache-airflow-providers-odbc==3.2.1
apache-airflow-providers-postgres==5.4.0
apache-airflow-providers-redis==3.1.0
apache-airflow-providers-sendgrid==3.1.0
apache-airflow-providers-sftp==4.2.4
apache-airflow-providers-slack==7.2.0
apache-airflow-providers-snowflake==4.0.5
apache-airflow-providers-sqlite==3.3.2
apache-airflow-providers-ssh==3.6.0
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else
After upgrading from 2.5.1 to 2.6.0
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct