airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dan Davydov (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-774) dagbag_size stat incorrect
Date Fri, 20 Jan 2017 00:04:26 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dan Davydov updated AIRFLOW-774:
--------------------------------
    Description: After the multiprocessor change was made (dag folders are processed in parallel),
the number of dags reported by airflow is for each of these subprocesses which is inaccurate,
and potentially orders of magnitude less than the actual number of dags. These individual
processes stats should be aggregated. The collect_dags/dagbag_import_errors stats should also
be fixed (time it takes to parse the dags).  (was: After the multiprocessor change was made
(dag folders are processed in parallel), the number of dags reported by airflow is for each
of these subprocesses which is inaccurate, and potentially orders of magnitude less than the
actual number of dags. These individual processes stats should be aggregated. The collect_dags
stat should also be fixed (time it takes to parse the dags).)

> dagbag_size stat incorrect
> --------------------------
>
>                 Key: AIRFLOW-774
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-774
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: logging
>            Reporter: Dan Davydov
>
> After the multiprocessor change was made (dag folders are processed in parallel), the
number of dags reported by airflow is for each of these subprocesses which is inaccurate,
and potentially orders of magnitude less than the actual number of dags. These individual
processes stats should be aggregated. The collect_dags/dagbag_import_errors stats should also
be fixed (time it takes to parse the dags).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message