airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jack (Jira)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-3344) Airflow DAG object clear function does not clear tasks in the upstream_failed state when only_failed=True
Date Sun, 17 Nov 2019 11:56:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-3344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16975993#comment-16975993
] 

jack commented on AIRFLOW-3344:
-------------------------------

[~steveatbat] can you PR your suggested change?

> Airflow DAG object clear function does not clear tasks in the upstream_failed state when
only_failed=True
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-3344
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3344
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: DAG
>    Affects Versions: 1.8.2, 1.9.0, 1.10.0
>            Reporter: Steve Jacobs
>            Priority: Minor
>              Labels: easyfix, newbie
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> When using the airflow clear command from the cli, you can pass an argument --only_failed
to clear only failed tasks. This will clear ONLY tasks with the state failed, and not tasks
with the state upstream_failed, causing any clear to still fail the dag_run if any upstream
tasks are failed.
> Since one_failed as a trigger rule also checks for upstream_failed tasks, it seems consistent
that this should also clear upstream_failed tasks. The relevant code change necessary is here:
> {code:java}
> if only_failed:
>  tis = tis.filter(TI.state == State.FAILED)
> {code}
> to
> {code:java}
> if only_failed:
>   tis = tis.filter(TI.state.in_([State.FAILED, State.UPSTREAM_FAILED]))
> {code}
> in models.py
> Additionally when clearing dags, the dag_run is set to the running state, but the dag_run
start_date is not updated to the current time, as it is when clearing tasks through the Web
UI. This causes dag_runs to fail on their timeouts even if the dag is full of successful tasks.
This needs to be changed as well.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message