airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shaw, Damian P. " <>
Subject RE: Outage report
Date Fri, 16 Aug 2019 15:52:53 GMT
Thanks, this is really useful to know!  I often write my own Operators/Sensors/Hooks and was
just looking at doing the same with the SFTPSensor and Operator.

I've never formalized it but my current pattern is the follow:

Set self._conn to None on __init__, and have a property "self.conn" that checks if "self._conn"
is None,
 *if None create a new connection set it to self._conn and return it
 * if not None run a check to see if the connection is still alive, if is alive return self._conn,
otherwise create a new connection 

On  __init__ set self.conn_id to the conn_id string, and set "self._{conn_type}_hook" to None
and have a property "self.{conn_type}_hook"
In property check if "self._{conn_type}_hook" is None and if so create a new Hook, if not
None then return "self._{conn_type}_hook"

I would be really appreciative on any  best practices here others could share. 

-----Original Message-----
From: James Meickle [] 
Sent: Friday, August 16, 2019 11:27 AM
Subject: Outage report

We had an outage last night that was rather complex and difficult to debug.
Rather than just writing up the bug, I included what we did for various
debug steps. Hope some folks who are also cluster maintainers may find it

Please access the attached hyperlink for an important electronic communications disclaimer: 

View raw message