airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ash Berlin-Taylor (Jira)" <j...@apache.org>
Subject [jira] [Resolved] (AIRFLOW-6175) Executor changes queued tasks to scheduled without removing from task queue
Date Tue, 10 Dec 2019 13:51:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ash Berlin-Taylor resolved AIRFLOW-6175.
----------------------------------------
    Fix Version/s:     (was: 1.10.6)
                   1.10.7
       Resolution: Fixed

> Executor changes queued tasks to scheduled without removing from task queue
> ---------------------------------------------------------------------------
>
>                 Key: AIRFLOW-6175
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6175
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 1.10.6
>            Reporter: Daniel Imberman
>            Priority: Blocker
>             Fix For: 1.10.7
>
>
> There is a bug caused by scheduler_jobs refactor which leads to task failure and scheduler
locking.
> Essentially when a there is an overflow of tasks going into the scheduler, the tasks
are set back to scheduled, but are not removed from the executor's queued_tasks queue.
> This means that the executor will attempt to run tasks that are in the scheduled state,
but those tasks will fail dependency checks. Eventually the queue is filled with scheduled
tasks, and the scheduler can no longer run.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message