airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kim Caffeinism (Jira)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-5654) Make 'OperatorSequence' to flexible operator deps define
Date Mon, 14 Oct 2019 07:23:00 GMT
Kim Caffeinism created AIRFLOW-5654:
---------------------------------------

             Summary: Make 'OperatorSequence' to flexible operator deps define
                 Key: AIRFLOW-5654
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5654
             Project: Apache Airflow
          Issue Type: New Feature
          Components: operators
    Affects Versions: 1.10.5
            Reporter: Kim Caffeinism


USAGE

x, y, z, a is some operators

 

seq1 = x >> y

seq2 = z >> seq1 >> a

 

now deps defined 

[x, z] -> y -> a

But this definition seems more intuitive

z >> x >> y >> a

 

code will be like this

 

class OperatorSequence:
 ....def __init__(self, head, tail):
 ........self.head = head
 ........self.tail = tail
 
 ....def __rshift__(self, other):
 ........self >> other
 ........return sequence_operator(self.head, other)
 
 ....def __lshift__(self, other):
 ........self << other
 ........return sequence_operator(other, self.tail)
 
 ....def __rrshift__(self, other):
 ........other >> self
 ........return sequence_operator(other, self.tail)
 
 ....def __rlshift__(self, other):
 ........other << self
........return sequence_operator(self.head, other)

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message