airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-1142) SubDAG Tasks Not Executed Even Though All Dependencies Met
Date Thu, 27 Apr 2017 19:18:04 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15987337#comment-15987337
] 

ASF subversion and git services commented on AIRFLOW-1142:
----------------------------------------------------------

Commit 5800f565628d11d8ea504468bcc14c4d1c0da10c in incubator-airflow's branch refs/heads/v1-8-test
from [~bolke]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=5800f56 ]

[AIRFLOW-1142] Do not reset orphaned state for backfills

The scheduler could interfere with backfills when
it resets the state
of tasks that were considered orphaned. This patch
prevents the scheduler
from doing so and adds a guard in the backfill.

Closes #2260 from bolkedebruin/AIRFLOW-1142

(cherry picked from commit 4e79b830e3261b9d54fdbc7c9dcb510d36565986)
Signed-off-by: Bolke de Bruin <bolke@xs4all.nl>


> SubDAG Tasks Not Executed Even Though All Dependencies Met
> ----------------------------------------------------------
>
>                 Key: AIRFLOW-1142
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1142
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: subdag
>    Affects Versions: 1.8.1
>         Environment: 1.8.1rc1+incubating, Celery
>            Reporter: Joe Schmid
>            Priority: Blocker
>             Fix For: 1.8.1
>
>         Attachments: 2017-04-24T23-20-38-776547, run3-scheduler-stdout.log, run3-task.log,
SubDAGOperatorTaskLog-DEBUG.txt, Test_Nested_SubDAG_0.png, Test_Nested_SubDAG_1-Zoomed.png,
test_nested_subdag.py
>
>
> Testing on 1.8.1rc1, we noticed that tasks in subdags were not getting executed even
though all dependencies had been met.
> We were able to create a simple test DAG that re-creates the issue. Attached is a test
DAG, the log file of the subdag operator that shows it fails to run even though dependencies
are met, and screenshots of what the UI looks like.
> This is definitely a regression as we have many similarly constructed DAGs that have
been running successfully on a pre-v1.8 version (a fork of 1.7.1.3+master) for some time.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message