airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-2203) Dag import too slow for very large dags
Date Wed, 14 Mar 2018 08:16:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398224#comment-16398224
] 

ASF subversion and git services commented on AIRFLOW-2203:
----------------------------------------------------------

Commit 6f0a0d2f6f8203d2c6542a8eaefe2cbb2640a8d3 in incubator-airflow's branch refs/heads/master
from [~wongwill]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=6f0a0d2 ]

[AIRFLOW-2203] Remove Useless Commands.

self.tasks is a temp list gen from self.task_dict. no reason to append
to it


> Dag import too slow for very large dags
> ---------------------------------------
>
>                 Key: AIRFLOW-2203
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2203
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: DAG
>    Affects Versions: 1.8.1, 1.8.0, 1.9.0, 1.9.1
>            Reporter: Will Wong
>            Assignee: Will Wong
>            Priority: Major
>              Labels: performance, usability
>
> Dag import for large dags is too slow. It is very easy to timeout very large dags with
< 500 tasks.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message