airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-1298) Airflow Clear Command does not clear tasks in UPSTREAM_FAILED state
Date Tue, 31 Jul 2018 05:35:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563151#comment-16563151
] 

ASF GitHub Bot commented on AIRFLOW-1298:
-----------------------------------------

ubermen opened a new pull request #3667: [AIRFLOW-1298] Add clear option 'only_failed_or_upstream_failed'
URL: https://github.com/apache/incubator-airflow/pull/3667
 
 
   ### Description
   In my operations team, it is necessary to clear all tasks in one command line (because
of so many schedules will be reprocessed)
   But, 'clear -cdf ...' is only clearing failed tasks without upstream_failed. It is not
fit for our operator's needs.
   So, I want to add new options to clear failed or upstream_failed jobs all at once.
   
   
   ### Tests
   1. clear only one schedule of this task : airflow clear -cp -s 2018-07-22T05:00:00 -e 2018-07-22T05:00:00
-t ^task_name$ schedule_name
   2. clear only multiple schedules of this task : airflow clear -cp -s 2018-07-22T05:00:00
-e 2018-07-22T09:00:00 -t ^task_name$ schedule_name
   3. clear multiple schedules of this task with downstream : airflow clear -cdp -s 2018-07-22T05:00:00
-e 2018-07-22T09:00:00 -t ^task_name$ schedule_name
   
   
   ### Documentation
   new option of clear command
   option is 'p' of upstream_failed's p character.
   Usage sample is like below :
   airflow clear -cdp -s 2018-07-22T05:00:00 -e 2018-07-22T09:00:00 -t ^task_name$ schedule_name

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Airflow Clear Command does not clear tasks in UPSTREAM_FAILED state
> -------------------------------------------------------------------
>
>                 Key: AIRFLOW-1298
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1298
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: cli
>    Affects Versions: Airflow 1.8
>         Environment: Ubuntu 12.04, Kernel: 3.13.0-113-generic, python 2.7.10
>            Reporter: Aaditya Ramesh
>            Assignee: Aaditya Ramesh
>            Priority: Critical
>
> We are unable to clear airflow tasks that are in the UPSTREAM_FAILED state using the
command line. The fix is simple - just change `clear` function in models.py to also clear
tasks in UPSTREAM_FAILED state, not just FAILED.
> Diff:
> {noformat}
> diff --git a/airflow/models.py b/airflow/models.py
> index 30e18a44..e60d2918 100755
> --- a/airflow/models.py
> +++ b/airflow/models.py
> @@ -3180,7 +3180,7 @@ class DAG(BaseDag, LoggingMixin):
>          if end_date:
>              tis = tis.filter(TI.execution_date <= end_date)
>          if only_failed:
> -            tis = tis.filter(TI.state == State.FAILED)
> +            tis = tis.filter(TI.state == State.FAILED or TI.state == State.UPSTREAM_FAILED)
>          if only_running:
>              tis = tis.filter(TI.state == State.RUNNING)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message