airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Anand (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-54) Special tags when referring to other tasks
Date Fri, 06 May 2016 17:31:12 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-54?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Siddharth Anand updated AIRFLOW-54:
-----------------------------------
    Description: 
At the moment when you need to refer to other tasks, you either have to refer to them by name
or search though a list of upstream objects. 

For example, in order to get the task_ids needed by xcom_pull for the upstream items, you
need to do:
    
     ti.xcom_pull(task_ids=[task.task_id for task in self.upstream_list])[0]

This issue suggests adding a keyword operator similar in form to those the schedule would
use. 

Possible items might include:
@upstream: Tasks directly upstream <--- *change this to @parent*
@ancestor: Any task that is in the ancestor tree of this dag
@sibling: other tasks stemming from the same upstream tasks
@children: tasks from the children of this node


Aside from providing more simple arguments, this also improves the readability of the functions.

  was:
At the moment when you need to refer to other tasks, you either have to refer to them by name
or search though a list of upstream objects. 

For example, in order to get the task_ids needed by xcom_pull for the upstream items, you
need to do:
    
     ti.xcom_pull(task_ids=[task.task_id for task in self.upstream_list])[0]

This issue suggests adding a keyword operator similar in form to those the schedule would
use. 

Possible items might include:
@upstream: Tasks directly upstream
@ancestor: Any task that is in the ancestor tree of this dag
@sibling: other tasks stemming from the same upstream tasks
@children: tasks from the children of this node


Aside from providing more simple arguments, this also improves the readability of the functions.


> Special tags when referring to other tasks
> ------------------------------------------
>
>                 Key: AIRFLOW-54
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-54
>             Project: Apache Airflow
>          Issue Type: New Feature
>          Components: hooks, operators
>            Reporter: Adam Mustafa
>            Priority: Minor
>
> At the moment when you need to refer to other tasks, you either have to refer to them
by name or search though a list of upstream objects. 
> For example, in order to get the task_ids needed by xcom_pull for the upstream items,
you need to do:
>     
>      ti.xcom_pull(task_ids=[task.task_id for task in self.upstream_list])[0]
> This issue suggests adding a keyword operator similar in form to those the schedule would
use. 
> Possible items might include:
> @upstream: Tasks directly upstream <--- *change this to @parent*
> @ancestor: Any task that is in the ancestor tree of this dag
> @sibling: other tasks stemming from the same upstream tasks
> @children: tasks from the children of this node
> Aside from providing more simple arguments, this also improves the readability of the
functions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message