airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeremiah Lowin (JIRA)" <>
Subject [jira] [Created] (AIRFLOW-1005) Speed up Airflow startup time
Date Fri, 17 Mar 2017 22:04:42 GMT
Jeremiah Lowin created AIRFLOW-1005:

             Summary: Speed up Airflow startup time
                 Key: AIRFLOW-1005
             Project: Apache Airflow
          Issue Type: Improvement
    Affects Versions: 1.8.0
            Reporter: Jeremiah Lowin
            Assignee: Jeremiah Lowin
            Priority: Minor

Airflow takes approximately 1 second to import. It turns out that more than half the time
is spend doing two things: importing Cryptography to create Fernet keys and importing Alembic.

The first Cryptography import is in and is only necessary if Airflow is generating
a new airflow.cfg file (but currently gets run every time). Therefore it can be easily deferred.

The second is in to check if encryption is turned on. This can also be deferred
until encryption checks are actually needed.

Alembic is always imported even though it is only needed for running initdb() and upgradedb().
It can be lazily imported inside those functions.

These simple changes reduce Airflow's startup time by half on my machine.

This message was sent by Atlassian JIRA

View raw message