airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julien Debbia (Jira)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-5400) "queue" parameter can't interpret Jinja2 templated value
Date Wed, 04 Sep 2019 07:34:00 GMT
Julien Debbia created AIRFLOW-5400:
--------------------------------------

             Summary: "queue" parameter can't interpret Jinja2 templated value
                 Key: AIRFLOW-5400
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5400
             Project: Apache Airflow
          Issue Type: Bug
          Components: operators, xcom
    Affects Versions: 1.10.2
            Reporter: Julien Debbia


Hello,

I'm trying to manage dynamically the queue value by defining it with an custom operator that
push a XCOM with the correct value (example: "queue1").

When I try to pull it from another operator (custom or native) with the parameter "queue",
it never interpret it and in the Task instance details, instead of the string value, I can
see the templated value (for example: \{{ ti.xcom_pull(tasks_id="get_queue") }} instead of
"queue1").

I added a BashOperator to echo the xcom value and everything is fine, I can see the value
"queue1".

This is a very big issue for me because I based all my development on this and it is not working.

In the documentation of the BaseOperator, the queue parameter documentation indicates that
only CeleryExecutor can handle queue dynamically, so I switched from SequentialExecutor to
CeleryExecutor but without success.

 

I asked on Slack about it and a nice guy told me that it is possible to pull a XCOM value
for the queue parameter and after many tries, I never succeed on pulling XCOM for this parameter.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Mime
View raw message