airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gerard Toonstra (JIRA)" <>
Subject [jira] [Assigned] (AIRFLOW-1143) Tasks rejected by workers get stuck in QUEUED
Date Thu, 27 Apr 2017 14:36:04 GMT


Gerard Toonstra reassigned AIRFLOW-1143:

    Assignee: Gerard Toonstra

> Tasks rejected by workers get stuck in QUEUED
> ---------------------------------------------
>                 Key: AIRFLOW-1143
>                 URL:
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>            Reporter: Dan Davydov
>            Assignee: Gerard Toonstra
> If the scheduler schedules a task that is sent to a worker that then rejects the task
(e.g. because one of the dependencies of the tasks became bad, like the pool became full),
the task will be stuck in the QUEUED state. We hit this trying to switch from invoking the
scheduler "airflow scheduler -n 5" to just "airflow scheduler".
> Restarting the scheduler fixes this because it cleans up orphans, but we shouldn't have
to restart the scheduler to fix these problems (the missing job heartbeats should make the
scheduler requeue the task).

This message was sent by Atlassian JIRA

View raw message