Mailing-List: contact commits-help@airflow.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airflow.incubator.apache.org
Date: Fri, 17 Mar 2017 22:04:42 +0000 (UTC)
From: "Jeremiah Lowin (JIRA)" <jira@apache.org>
To: commits@airflow.incubator.apache.org
Message-ID: <JIRA.13057173.1489788278000.54317.1489788282059@Atlassian.JIRA>
In-Reply-To: <JIRA.13057173.1489788278000@Atlassian.JIRA>
References: <JIRA.13057173.1489788278000@Atlassian.JIRA> <JIRA.13057173.1489788278020@jira-lw-us.apache.org>
Subject: [jira] [Created] (AIRFLOW-1005) Speed up Airflow startup time
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 17 Mar 2017 22:04:48 -0000

Jeremiah Lowin created AIRFLOW-1005:
---------------------------------------

             Summary: Speed up Airflow startup time
                 Key: AIRFLOW-1005
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1005
             Project: Apache Airflow
          Issue Type: Improvement
    Affects Versions: 1.8.0
            Reporter: Jeremiah Lowin
            Assignee: Jeremiah Lowin
            Priority: Minor


Airflow takes approximately 1 second to import. It turns out that more than half the time is spend doing two things: importing Cryptography to create Fernet keys and importing Alembic. 

The first Cryptography import is in configuration.py and is only necessary if Airflow is generating a new airflow.cfg file (but currently gets run every time). Therefore it can be easily deferred.

The second is in models.py to check if encryption is turned on. This can also be deferred until encryption checks are actually needed.

Alembic is always imported even though it is only needed for running initdb() and upgradedb(). It can be lazily imported inside those functions.

These simple changes reduce Airflow's startup time by half on my machine.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)