Mailing-List: contact commits-help@airflow.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airflow.incubator.apache.org
Date: Fri, 26 May 2017 18:59:04 +0000 (UTC)
From: "Dan Davydov (JIRA)" <jira@apache.org>
To: commits@airflow.incubator.apache.org
Message-ID: <JIRA.13075246.1495824201000.304960.1495825144997@Atlassian.JIRA>
In-Reply-To: <JIRA.13075246.1495824201000@Atlassian.JIRA>
References: <JIRA.13075246.1495824201000@Atlassian.JIRA> <JIRA.13075246.1495824201561@jira-lw-us.apache.org>
Subject: [jira] [Updated] (AIRFLOW-1249) Running tasks from backfills with
 unmet conditions are stuck running
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 26 May 2017 18:59:10 -0000


     [ https://issues.apache.org/jira/browse/AIRFLOW-1249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dan Davydov updated AIRFLOW-1249:
---------------------------------
    Description: 
Running tasks from backfills with unmet conditions are stuck running. E.g. create a DAG and pause it, mark the dagrun for a specific date as failed in the UI, and then start a backfill for that date. Note that the task is stuck in the running or queued state with no start/end date set. I believe this occurs for the Celery executor but it may affect other/all executors.

This can e.g. cause pools to fill up.

This query cleans up the tasks as a workaround:
{code}
DELETE ti FROM task_instance ti
        JOIN dag_run dr
                ON ti.execution_date = dr.execution_date AND
           ti.dag_id = dr.dag_id
           JOIN dag dg
           ON dr.dag_id=dg.dag_id
        WHERE ISNULL(ti.start_date) and (ti.state="queued" or ti.state="running")
        and dr.state="failed" and (dg.is_paused="1" or dg.is_active="0");
{code}

  was:
Running tasks from backfills with unmet conditions are stuck running. E.g. create a DAG and pause it, mark the dagrun for a specific date as failed in the UI, and then start a backfill for that date. Note that the task is stuck in the running or queued state with no start/end date set. I believe this occurs for the Celery executor but it may affect other/all executors.

This can e.g. cause pools to fill up.

This query cleans up the tasks as a workaround:
{code}
DELETE ti FROM task_instance ti
        JOIN dag_run dr
                ON ti.execution_date = dr.execution_date AND
           ti.dag_id = dr.dag_id
           JOIN dag dg
           ON dr.dag_id=dg.dag_id
        WHERE ISNULL(ti.start_date) and (ti.state="queued" or ti.state="running")
        and dr.state="failed"  and dg.is_paused="1";
{code}


> Running tasks from backfills with unmet conditions are stuck running
> --------------------------------------------------------------------
>
>                 Key: AIRFLOW-1249
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1249
>             Project: Apache Airflow
>          Issue Type: Bug
>            Reporter: Dan Davydov
>
> Running tasks from backfills with unmet conditions are stuck running. E.g. create a DAG and pause it, mark the dagrun for a specific date as failed in the UI, and then start a backfill for that date. Note that the task is stuck in the running or queued state with no start/end date set. I believe this occurs for the Celery executor but it may affect other/all executors.
> This can e.g. cause pools to fill up.
> This query cleans up the tasks as a workaround:
> {code}
> DELETE ti FROM task_instance ti
>         JOIN dag_run dr
>                 ON ti.execution_date = dr.execution_date AND
>            ti.dag_id = dr.dag_id
>            JOIN dag dg
>            ON dr.dag_id=dg.dag_id
>         WHERE ISNULL(ti.start_date) and (ti.state="queued" or ti.state="running")
>         and dr.state="failed" and (dg.is_paused="1" or dg.is_active="0");
> {code}


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)