airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adrian Bridgett (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-78) airflow clear leaves dag_runs
Date Tue, 17 May 2016 08:48:12 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15286234#comment-15286234
] 

Adrian Bridgett commented on AIRFLOW-78:
----------------------------------------

Sure (sorry for the delay). My understanding was that it "clears" it - so that it's as if
it's never run.  The scheduler then comes along and says "oh, I'd better run that then" and
runs the DAG (or task if you've just cleared a task within a DAG).   Back when I first used
airflow I expected "run" to force-run a task but that didn't seem to happen (even with "force"
selected) and the code matched that (I see that it should work now if the task isn't successful
or force is set now).

The specific issue I had here was to do with that max_active_runs setting - I think it must
have stopped any task from running at all (I don't think that the case that we had _two_ runs
simultaneously as it's unlikely I ran the clear when another run was running).  The fact that
I had to clear the dagrun seems to confirm this to me. 

> airflow clear leaves dag_runs
> -----------------------------
>
>                 Key: AIRFLOW-78
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-78
>             Project: Apache Airflow
>          Issue Type: Wish
>          Components: cli
>    Affects Versions: Airflow 1.6.2
>            Reporter: Adrian Bridgett
>            Assignee: Siddharth Anand
>            Priority: Minor
>
> (moved from https://github.com/apache/incubator-airflow/issues/829)
> "airflow clear -c -d -s 2016-01-03 dagid"  doesn't clear the dagrun, it sets it to running
instead (apparently since this is often used to re-run jobs).
> However this then breaks max_active_runs=1 (I have to stop the scheduler, then airflow
clear, psql to delete the dagrun, then start the scheduler).
> This problem was probably seen on an Airflow 1.6.x install.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message