Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Task with Branch Operator is following both branch #883

Open
2 tasks done
ramjeeanna opened this issue Sep 12, 2024 · 1 comment
Open
2 tasks done

Task with Branch Operator is following both branch #883

ramjeeanna opened this issue Sep 12, 2024 · 1 comment
Labels
kind/bug kind - things not working properly lifecycle/stale lifecycle - this is stale

Comments

@ramjeeanna
Copy link

Checks

Chart Version

airflow-8.9.0

Kubernetes Version

1.28.9

Client Version: v1.31.0
Kustomize Version: v5.4.2
Server Version: v1.28.9
WARNING: version difference between client (1.31) and server (1.28) exceeds the supported minor version skew of +/-1

Helm Version

version.BuildInfo{Version:"v3.14.2", GitCommit:"", GitTreeState:"clean", GoVersion:"go1.21.11"}

Description

When multiple DAG run is happening, sometimes the kwargs['param'] value are malfunctioned.

create Airflow DAG with Task Branch operator like below
pre_check = BranchPythonOperator(task_id='pre_validlate',
python_callable=pre_validate_fn).
graph is below:
init_report >> pre_check >> data_extraction >> end
pre_check >> remove_table

The function of branch operator, pre_validate_fn will decide which branch to follow, the code is below:
follow_branch = 'data_extraction'
if not kwargs['param']['delta']:
follow_branch = ['data_extraction', 'remove_table']
print(follow_branch)
return follow_branch
In we dont set detla, then it will remove as well as go for extraction
In our case, if our input parameter is passed wit 'delta' as True even in that case, the condition is followed.

When such malfunctioned execution happens, the task remove_table is shown success in UI and table also deleted in logic. But the respective task_id=remove_table logs folder is not created under the relevant run_id folder.

Relevant Logs

NA

Custom Helm Values

NA
@ramjeeanna ramjeeanna added the kind/bug kind - things not working properly label Sep 12, 2024
Copy link

stale bot commented Jan 31, 2025

This issue has been automatically marked as stale because it has not had activity in 60 days.
It will be closed in 7 days if no further activity occurs.

Thank you for your contributions.


Issues never become stale if any of the following is true:

  1. they are added to a Project
  2. they are added to a Milestone
  3. they have the lifecycle/frozen label

@stale stale bot added the lifecycle/stale lifecycle - this is stale label Jan 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug kind - things not working properly lifecycle/stale lifecycle - this is stale
Projects
None yet
Development

No branches or pull requests

1 participant