airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Huang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (AIRFLOW-872) Tasks are not skipping correctly
Date Wed, 22 Feb 2017 17:19:44 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876942#comment-15876942
] 

Daniel Huang edited comment on AIRFLOW-872 at 2/22/17 5:19 PM:
---------------------------------------------------------------

I hit this as well with the LatestOnlyOperator (very similar to ShortCircuitOperator, it also
sets its direct downstream tasks to SKIPPED) when there are more than one task downstream
from the latest only task.

I believe this [block|https://github.com/apache/incubator-airflow/blob/50702d06187035c99e51ea936c756c00332c4a4a/airflow/models.py#L4034]
is being hit and setting the dag to failed prematurely. When I commented it out as a temporary
workaround, the remaining tasks get set to skipped correctly and the dag ends in success.


was (Author: dxhuang):
I hit this as well with the LatestOnlyOperator (very similar to ShortCircuitOperator, it also
sets its direct downstream tasks to SKIPPED) when there are more than one task downstream
from the latest only task.

I believe this [block](https://github.com/apache/incubator-airflow/blob/50702d06187035c99e51ea936c756c00332c4a4a/airflow/models.py#L4034)
is being hit and setting the dag to failed prematurely. When I commented it out as a temporary
workaround, the remaining tasks get set to skipped correctly and the dag ends in success.

> Tasks are not skipping correctly
> --------------------------------
>
>                 Key: AIRFLOW-872
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-872
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: cli, DagRun, webserver
>    Affects Versions: Airflow 1.8
>         Environment: docker-compose using docker-compose-LocalExecutor.yml  on this fork/branch
https://github.com/fdm1/docker-airflow/tree/v1-8-not-skipping
>            Reporter: Frank Massi
>         Attachments: Screen Shot 2017-02-13 at 1.09.35 PM.png, Screen Shot 2017-02-13
at 1.11.09 PM.png, Screen Shot 2017-02-13 at 1.11.15 PM.png, Screen Shot 2017-02-13 at 1.13.13
PM.png
>
>
> When using the BranchPythonOperator or ShortCircuitOperator to make Dags idempotent,
if running the Dag after a successful DagRun and clearing the initial task, the second task
skips and then the Dag enters a failed state, as opposed to skipping all remaining tasks and
marking the Dag successful.
> Steps to reproduce are outlined here: https://github.com/fdm1/docker-airflow/blob/v1-8-not-skipping/dags/test_clear_bug.py#L4-L9
> In 1.7.1.3, the tasks do skip correctly, so this appears to be a new bug.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message