Mailing-List: contact commits-help@airflow.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airflow.incubator.apache.org
Date: Wed, 2 Aug 2017 22:07:00 +0000 (UTC)
From: "Ace Haidrey (JIRA)" <jira@apache.org>
To: commits@airflow.incubator.apache.org
Message-ID: <JIRA.13036383.1484870509000.78504.1501711620632@Atlassian.JIRA>
In-Reply-To: <JIRA.13036383.1484870509000@Atlassian.JIRA>
References: <JIRA.13036383.1484870509000@Atlassian.JIRA> <JIRA.13036383.1484870509475@jira-lw-us.apache.org>
Subject: [jira] [Commented] (AIRFLOW-774)
 dagbag_size/collect_dags/dagbag_import_errors stats incorrect
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 02 Aug 2017 22:07:06 -0000


    [ https://issues.apache.org/jira/browse/AIRFLOW-774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111805#comment-16111805 ] 

Ace Haidrey commented on AIRFLOW-774:
-------------------------------------

Are those changes merged in then? To the master branch or anywhere?
And to clarify so you fixed the import errors are fixed but the correcting of the stats are not corrected yet, is that right?


> dagbag_size/collect_dags/dagbag_import_errors stats incorrect
> -------------------------------------------------------------
>
>                 Key: AIRFLOW-774
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-774
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: logging
>            Reporter: Dan Davydov
>
> After the multiprocessor change was made (dag folders are processed in parallel), the number of dags reported by airflow is for each of these subprocesses which is inaccurate, and potentially orders of magnitude less than the actual number of dags. These individual processes stats should be aggregated. The collect_dags/dagbag_import_errors stats should also be fixed (time it takes to parse the dags).


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)