airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amit Jain <aj201...@gmail.com>
Subject Task Priority per schedule run or applied on entire queue
Date Mon, 06 Mar 2017 18:16:54 GMT
Hi All,

I have a doubt related to task instance priority. When we specify the
priority_weight on the operator, does task priority re-ordering happens
between the eligible task instances at the time of Scheduler run or does it
re-order queue in AMQP supported broker (RabbitMQ supports priority_queue
3.5.0 onwards)?

I think first case is happening here. Here is code from the master branch.

// jobs.py
def _execute_task_instances(....):

priority_sorted_task_instances = sorted(
                task_instances, key=lambda ti: (-ti.priority_weight,
ti.execution_date))



--
Thanks,
Amit

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message