Replies: 2 comments 4 replies
-
|
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval. |
Beta Was this translation helpful? Give feedback.
-
|
I guess you are talking about "mini-scheduler" only - your downstream tasks are still getting scheduled but not immediately after task completes. Your downstream tasks will be scheduled by the next scheduler loop I presume. If so. Yes. It might happen. "Mini-scheduler" is an optimisation that is not guaranteed to work. In fact in Airlfow 3 we are removing it altogether because it has proven to be problematic in many ways - one of the problems was that it causes excessive locking on the database - where the database can be locked by - sometimes - many tasks completing at about the same time. So I believe in Airflow 2.8 we implemented a strategy to skip some of that if the contention is too high. Also this has been further improved in 2.9 and 2.10 so you will likely get a bit better behaviour when you upgrade. But still - there is no guarantee it will happen immediately after task completes. |
Beta Was this translation helpful? Give feedback.
-
|
Hello, in fact, it will not schedule downstream tasks all the time, we have been verified in production, the probability of production in the near future is very high, we use the executor CeleryExecutor, currently open 2 schedulers and 3 workers, is there an effective way for us to solve this problem, thank you very much |
Beta Was this translation helpful? Give feedback.
-
|
No idea. If your task is not scheduled, this is strange. you need to look and provide more information and details and hope someone will be able to figure out what's wrong - the logs you shown were from "mini-scheduler" - so look for related scheduler logs and see if you see anything strange. You can also disable mini-scheduler to remove it from the picture: https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#schedule-after-task-execution maybe that will help as "mini-scheduler" had various side effects in some edge cases and it helped people in a number of cases in the past. |
Beta Was this translation helpful? Give feedback.
-
|
Okay, thank you very much for your answer, let's look into this issue ourselves again |
Beta Was this translation helpful? Give feedback.
-
|
Hello, I found a new clue, here is running but the state of the downstream node has not changed, is there a concurrency problem in your scheduling logic, resulting in the state is not updated in time, will this be the clue to the above problem |
Beta Was this translation helpful? Give feedback.
-
Apache Airflow version
2.10.4
If "Other Airflow 2 version" selected, which one?
2.8.2
What happened?
What you think should happen instead?
Looking at the source code, it seems that the logic of the scheduling check block has just been modified on October 23rd.,There's a labor gang to take a look.
How to reproduce
I can't reproduce it for the time being, this problem is occasional, it should be a problem in the code logic
Operating System
Ubuntu
Versions of Apache Airflow Providers
No response
Deployment
Official Apache Airflow Helm Chart
Deployment details
Airflow 2.10.4 deployed by K8S + Helm
Anything else?
No response
Are you willing to submit PR?
Code of Conduct
Beta Was this translation helpful? Give feedback.
All reactions