airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-1641) Task gets stuck in queued state
Date Fri, 20 Oct 2017 19:12:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16213078#comment-16213078
] 

Andrew Loughran commented on AIRFLOW-1641:
------------------------------------------

Great to see some comments on this issue - thanks [~nickjbauer]!

I agree, and may raise a subsequent bug if I can reproduce it for a timed-out task to go into
a failed state, rather than end up being stuck.  IF this is the case, it's hopefull a case
of handling that state management a bit tighter.

Cheers!

> Task gets stuck in queued state
> -------------------------------
>
>                 Key: AIRFLOW-1641
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1641
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 1.8.0
>         Environment: Linux
>            Reporter: Mas
>              Labels: queued, scheduler, stuck, task
>
> Hello,
> I have one dag with ~20 tasks. 
> The dags runs daily and some tasks can sometime last for hours, depending on the processed
data behind.
> There are some interactions with AWS and a remote DB.
> I only use LocalExecutor.
> What this issue is about, is the fact that sometime (randomly, and without any clear
reason) one of the tasks (here also, it is random) gets stuck in "queued" state and never
starts running. 
> The manual workaround is to restart the task manually by clearing it.
> Does anyone have ideas about the issue behind, and how to avoid it for the future? 
> Thanks in advance for your help.
> PS: other people are facing the same behaviour: [link|https://stackoverflow.com/questions/45853013/airflow-tasks-get-stuck-at-queued-status-and-never-gets-running]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message