airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [airflow] ashb commented on issue #6760: [AIRFLOW-6157] Separate out common protocol for executors.
Date Tue, 10 Dec 2019 12:04:41 GMT
ashb commented on issue #6760: [AIRFLOW-6157] Separate out common protocol for executors.
URL: https://github.com/apache/airflow/pull/6760#issuecomment-564002923
 
 
   > > I do not fully understand why we need this? Can you explain the difference between
this protocol and the original base class?
   > 
   > The main problem is that in a number of places in the code (for example here: [3784ba4#diff-f95b3d6216459574a0b3954a85a98db1L251](https://github.com/apache/airflow/commit/3784ba4ce5a4618d7e090adfd4130e1d64fc812d#diff-f95b3d6216459574a0b3954a85a98db1L251)
) other entities (for example base_job) used internal implementation of BaseExecutor (it assumed
that some internal queues are there and that they are accessible as executor queued_tasks.
Similar was with "running". This does not have to be the case. For example in case of multiple
executors there might be multiple queues and it is much better to define the interface as
method rather than property being queue.
   > 
   > It's just plain old encapsulation in the works. So we have now `executor.is_task_queued(self,
task_instance_key: TaskInstanceKeyType))` instead of `task_instance_key in executor.queued`.
   > 
   > This is not strictly needed now, but if we define other executors in the future, we
might then abstract away from implementing those internal structures differently. And I already
tested and had it implemented in MultipleExecutors case so i know it's a workable change.
   
   The changes make sense, but just not the addition of the extra "Protocol" base class.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message