airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [airflow] jaketf commented on issue #6210: [AIRFLOW-5567] [Do not Merge] prototype BaseAsyncOperator
Date Thu, 03 Oct 2019 03:16:23 GMT
jaketf commented on issue #6210: [AIRFLOW-5567] [Do not Merge] prototype BaseAsyncOperator
URL: https://github.com/apache/airflow/pull/6210#issuecomment-537766761
 
 
   I have noticed in looking at existing dataflow and  dataproc hooks many hooks conflate
job submission and `wait_for_done` logic so migrating an existing operator to expose an async
capability often requires a decent leg work on the hook side.
   I imagine this is more than a coincidence. Is their guiding principles / design patterns
for hooks? In a world where we have a way to support an async operator it seems it'd be good
to have guiding principles for "functions should do one thing" in hooks rather than "this
function submits a job and waits for it to be done"

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message