airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Attila Kevei (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-1646) UI Tree View crashes due to vast amount of data generated by backend
Date Wed, 27 Sep 2017 09:40:00 GMT
Attila Kevei created AIRFLOW-1646:
-------------------------------------

             Summary: UI Tree View crashes due to vast amount of data generated by backend
                 Key: AIRFLOW-1646
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1646
             Project: Apache Airflow
          Issue Type: Bug
          Components: ui, webserver
    Affects Versions: Airflow 1.8
         Environment: CentOS 7, Python 2.7
            Reporter: Attila Kevei


Given a DAG with ~40 tasks that are mostly sequential but there are a couple of side branches.
A flow usually contains 20-24 tasks.
Airflow webserver's UI Tree View crashes on browsers with the default {{num_runs=25}} setting.
By investigating the issue it turned out that the backend produced a vast amount of HTML since
unnecessarily much JavaScript data was copied into the markup. Due to this the browser crashes
when tries to render it. Smaller {{num_runs}} can still be rendered (but {{num_runs=20}} requires
minutes if it works at all).
Some info on how much HTML content is generated at several {{num_runs}} values: {{num_runs=5
=> 90MB, num_runs=10 => 169MB, num_runs=15 => 248MB, num_runs=20 => 315MB, num_runs=25
=> 381MB}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message