airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adam Mustafa (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-54) Special tags when referring to other tasks
Date Fri, 06 May 2016 04:01:12 GMT
Adam Mustafa created AIRFLOW-54:
-----------------------------------

             Summary: Special tags when referring to other tasks
                 Key: AIRFLOW-54
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-54
             Project: Apache Airflow
          Issue Type: New Feature
          Components: hooks, operators
            Reporter: Adam Mustafa
            Priority: Minor


At the moment when you need to refer to other tasks, you either have to refer to them by name
or search though a list of upstream objects. 

For example, in order to get the task_ids needed by xcom_pull for the upstream items, you
need to do:
    
     ti.xcom_pull(task_ids=[task.task_id for task in self.upstream_list])[0]

This issue suggests adding a keyword operator similar in form to those the schedule would
use. 

Possible items might include:
@upstream: Tasks directly upstream
@ancestor: Any task that is in the ancestor tree of this dag
@sibling: other tasks stemming from the same upstream tasks
@children: tasks from the children of this node


Aside from providing more simple arguments, this also improves the readability of the functions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message