airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bolke de Bruin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (AIRFLOW-56) Airflow's scheduler can "loose" queued tasks
Date Fri, 06 May 2016 12:24:13 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-56?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273972#comment-15273972
] 

Bolke de Bruin edited comment on AIRFLOW-56 at 5/6/16 12:23 PM:
----------------------------------------------------------------

Attached dag that creates the behavior


was (Author: bolke):
Dag that creates the behavior

> Airflow's scheduler can "loose" queued tasks
> --------------------------------------------
>
>                 Key: AIRFLOW-56
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-56
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: executor, scheduler
>    Affects Versions: Airflow 1.7.0
>            Reporter: Bolke de Bruin
>            Priority: Blocker
>             Fix For: Airflow 1.7.1
>
>         Attachments: dag-queue.py
>
>
> Tasks that get into queued status can get "lost" on the scheduler/executor. They remain
in the DB with state "queued", but the scheduler won't pick them up as it will not look at
the DB currently. The executor does not see those tasks, although the scheduler has NOT been
restarted (ie. to see this issue it is not required to have "-n x" as a parameter to the scheduler).
> This happens *also* to tasks scheduled by the scheduler and not just with backfills created
by the SubDagOperator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message