Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

XCom_pull of multiple task_ids not returning the list values #41983

Open
1 of 2 tasks
edulauer opened this issue Sep 3, 2024 · 3 comments
Open
1 of 2 tasks

XCom_pull of multiple task_ids not returning the list values #41983

edulauer opened this issue Sep 3, 2024 · 3 comments
Labels
area:core kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet

Comments

@edulauer
Copy link

edulauer commented Sep 3, 2024

Apache Airflow version

2.10.0

If "Other Airflow 2 version" selected, which one?

No response

What happened?

When passing a list of task IDs to the task_ids argument in task_instance.xcom_pull, instead of receiving a list of values returned by each task, you might encounter a string with the value 'LazySelectSequence'."
In older versions of Airflow, such as 2.9.3, this issue did not occur.

What you think should happen instead?

It should return a list of values pushed by each task mentioned in the task_ids argument

How to reproduce

with DAG as dag:

push_task = PythonOperator(
    task_id='push_task',
    python_callable=lambda: 'Hello, World!')

push_task_2 = PythonOperator(
    task_id='push_task_2',
    python_callable=lambda: 'Hello, World 2!')

def pull_function(**kwargs):
    ti = kwargs['ti']
    msg = ti.xcom_pull(task_ids=['push_task', 'push_task_2'])
    print("received message: '%s'" % msg)

pull_task = PythonOperator(
    task_id='pull_task',
    python_callable=pull_function,
    provide_context=True,
)

Operating System

Debian GNU/Linux 12

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

FROM apache/airflow:2.10.0-python3.10

Anything else?

In older versions of Airflow, such as 2.9.3, this issue did not occur.

Error log:
[2024-09-03, 17:48:50 -03] {logging_mixin.py:190} INFO - received message: 'LazySelectSequence([2 items])'

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@edulauer edulauer added area:core kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet labels Sep 3, 2024
Copy link

boring-cyborg bot commented Sep 3, 2024

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

@FelipeRamos-neuro
Copy link

FelipeRamos-neuro commented Sep 24, 2024

I recently ran into this same behavior using xcom_pull with jinja and can confirm that this is also happening in version 2.10.2.

@johncmerfeld
Copy link
Contributor

I believe the root cause of this is #39426 but I don't know if my recent PR (#43978) will fix it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:core kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet
Projects
None yet
Development

No branches or pull requests

3 participants