airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adam Mustafa (JIRA)" <>
Subject [jira] [Created] (AIRFLOW-54) Special tags when referring to other tasks
Date Fri, 06 May 2016 04:01:12 GMT
Adam Mustafa created AIRFLOW-54:

             Summary: Special tags when referring to other tasks
                 Key: AIRFLOW-54
             Project: Apache Airflow
          Issue Type: New Feature
          Components: hooks, operators
            Reporter: Adam Mustafa
            Priority: Minor

At the moment when you need to refer to other tasks, you either have to refer to them by name
or search though a list of upstream objects. 

For example, in order to get the task_ids needed by xcom_pull for the upstream items, you
need to do:
     ti.xcom_pull(task_ids=[task.task_id for task in self.upstream_list])[0]

This issue suggests adding a keyword operator similar in form to those the schedule would

Possible items might include:
@upstream: Tasks directly upstream
@ancestor: Any task that is in the ancestor tree of this dag
@sibling: other tasks stemming from the same upstream tasks
@children: tasks from the children of this node

Aside from providing more simple arguments, this also improves the readability of the functions.

This message was sent by Atlassian JIRA

View raw message