airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tao Feng (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (AIRFLOW-1104) Concurrency check in scheduler should count queued tasks as well as running
Date Thu, 22 Mar 2018 06:29:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tao Feng reassigned AIRFLOW-1104:
---------------------------------

    Assignee: Tao Feng

> Concurrency check in scheduler should count queued tasks as well as running
> ---------------------------------------------------------------------------
>
>                 Key: AIRFLOW-1104
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1104
>             Project: Apache Airflow
>          Issue Type: Bug
>         Environment: see https://github.com/apache/incubator-airflow/pull/2221
> "Tasks with the QUEUED state should also be counted below, but for now we cannot count
them. This is because there is no guarantee that queued tasks in failed dagruns will or will
not eventually run and queued tasks that will never run will consume slots and can stall a
DAG. Once we can guarantee that all queued tasks in failed dagruns will never run (e.g. make
sure that all running/newly queued TIs have running dagruns), then we can include QUEUED tasks
here, with the constraint that they are in running dagruns."
>            Reporter: Alex Guziel
>            Assignee: Tao Feng
>            Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message