airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edgar Rodriguez (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-1519) Main DAG list page does not scale using client side paging
Date Fri, 18 Aug 2017 00:18:02 GMT
Edgar Rodriguez created AIRFLOW-1519:
----------------------------------------

             Summary: Main DAG list page does not scale using client side paging
                 Key: AIRFLOW-1519
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1519
             Project: Apache Airflow
          Issue Type: Bug
          Components: ui
            Reporter: Edgar Rodriguez
            Assignee: Edgar Rodriguez


Airflow's main page with DAGs listing takes too long to load (> 10 secs) when scale in
number of DAGS increases to 1K+ DAGs.

Airflow's main page with DAGs listing performs client side paging, by loading all dags in
the system and loading them to the jquery.DataTable plugin, which is pretty slow processing
the elements within the DOM in the client's browser. Additionally, when there are 1K+ DAGs,
rendering them in the server side via flask templates also introduces some overhead.

Solution would be to do a server side paging and aligning the page size to the one configured
for the web server (see https://github.com/apache/incubator-airflow/commit/04bfba3aa97deab850c14763279d33a6dfceb205),
providing consistent paging across the system.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message