Mailing-List: contact commits-help@airflow.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airflow.incubator.apache.org
Date: Sun, 19 Mar 2017 20:30:41 +0000 (UTC)
From: "Joe Schmid (JIRA)" <jira@apache.org>
To: commits@airflow.incubator.apache.org
Message-ID: <JIRA.13057346.1489933675000.61011.1489955441776@Atlassian.JIRA>
In-Reply-To: <JIRA.13057346.1489933675000@Atlassian.JIRA>
References: <JIRA.13057346.1489933675000@Atlassian.JIRA> <JIRA.13057346.1489933675681@jira-lw-us.apache.org>
Subject: [jira] [Updated] (AIRFLOW-1011) Task Instance Results not stored
 for SubDAG Tasks
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Sun, 19 Mar 2017 20:30:52 -0000


     [ https://issues.apache.org/jira/browse/AIRFLOW-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joe Schmid updated AIRFLOW-1011:
--------------------------------
    Description: 
In previous Airflow versions, results for tasks executed as a subdag were written as rows to task_instances. In Airflow 1.8 only rows for tasks inside the top-level DAG (non-subdag tasks) seem to get written to the database.

This results in being unable to check the status of task instances inside the subdag from the UI, check the logs for those task instances from the UI, etc.

Attached is a simple test DAG that exhibits the issue along with screenshots showing the UI differences between v1.8 and v1.7.1.3.

Note that if the DAG is run via backfill from command line (e.g. "airflow backfill Test_SubDAG -s 2017-03-18 -e 2017-03-18") the task instances show up successfully.

Also, we're using CeleryExecutor and not specifying a different executor for our subdags.

  was:
In previous Airflow versions, results for tasks executed as a subdag were written as rows to task_instances. In Airflow 1.8 only rows for tasks inside the top-level DAG (non-subdag tasks) seem to get written to the database.

This results in being unable to check the status of task instances inside the subdag from the UI, check the logs for those task instances from the UI, etc.

Attached is a simple test DAG that exhibits the issue along with screenshots showing the UI differences between v1.8 and v1.7.1.3.

Also, we're using CeleryExecutor and not specifying a different executor for our subdags.


> Task Instance Results not stored for SubDAG Tasks
> -------------------------------------------------
>
>                 Key: AIRFLOW-1011
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1011
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: backfill, subdag
>    Affects Versions: Airflow 1.8
>            Reporter: Joe Schmid
>            Priority: Critical
>         Attachments: 1-TopLevelDAGTaskInstancesShownCorrectly.png, 2-ZoomedSubDAG-NoTaskInstances-v1.8.png, 3-ZoomedSubDAG-TaskInstances-v1.7.1.3.png, test_subdag.py
>
>
> In previous Airflow versions, results for tasks executed as a subdag were written as rows to task_instances. In Airflow 1.8 only rows for tasks inside the top-level DAG (non-subdag tasks) seem to get written to the database.
> This results in being unable to check the status of task instances inside the subdag from the UI, check the logs for those task instances from the UI, etc.
> Attached is a simple test DAG that exhibits the issue along with screenshots showing the UI differences between v1.8 and v1.7.1.3.
> Note that if the DAG is run via backfill from command line (e.g. "airflow backfill Test_SubDAG -s 2017-03-18 -e 2017-03-18") the task instances show up successfully.
> Also, we're using CeleryExecutor and not specifying a different executor for our subdags.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)