airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vijay Krishna Ramesh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-1011) Task Instance Results not stored for SubDAG Tasks
Date Mon, 20 Mar 2017 16:37:41 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15932997#comment-15932997
] 

Vijay Krishna Ramesh commented on AIRFLOW-1011:
-----------------------------------------------

Ah just to clarify, the behavior I detailed above is for a normally scheduled dag run, not
a backfill or manually triggered run via trigger_dag. 

I do recall some weirdness with backfill and sub dag task instances after upgrading to the
1.8 RC4 but I don't have specific notes or logs saved to see if it was the same issue you
are seeing. 

> Task Instance Results not stored for SubDAG Tasks
> -------------------------------------------------
>
>                 Key: AIRFLOW-1011
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1011
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: backfill, subdag
>    Affects Versions: Airflow 1.8
>            Reporter: Joe Schmid
>            Priority: Critical
>         Attachments: 1-TopLevelDAGTaskInstancesShownCorrectly.png, 2-ZoomedSubDAG-NoTaskInstances-v1.8.png,
3-ZoomedSubDAG-TaskInstances-v1.7.1.3.png, test_subdag.py
>
>
> In previous Airflow versions, results for tasks executed as a subdag were written as
rows to task_instances. In Airflow 1.8 only rows for tasks inside the top-level DAG (non-subdag
tasks) seem to get written to the database.
> This results in being unable to check the status of task instances inside the subdag
from the UI, check the logs for those task instances from the UI, etc.
> Attached is a simple test DAG that exhibits the issue along with screenshots showing
the UI differences between v1.8 and v1.7.1.3.
> Note that if the DAG is run via backfill from command line (e.g. "airflow backfill Test_SubDAG
-s 2017-03-18 -e 2017-03-18") the task instances show up successfully.
> Also, we're using CeleryExecutor and not specifying a different executor for our subdags.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message