airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeremiah Lowin (JIRA)" <>
Subject [jira] [Updated] (AIRFLOW-1005) Speed up Airflow startup time
Date Fri, 17 Mar 2017 22:05:41 GMT


Jeremiah Lowin updated AIRFLOW-1005:
    Attachment: Screen Shot 2017-03-17 at 6.04.23 PM.png

> Speed up Airflow startup time
> -----------------------------
>                 Key: AIRFLOW-1005
>                 URL:
>             Project: Apache Airflow
>          Issue Type: Improvement
>    Affects Versions: 1.8.0
>            Reporter: Jeremiah Lowin
>            Assignee: Jeremiah Lowin
>            Priority: Minor
>         Attachments: Screen Shot 2017-03-17 at 6.04.23 PM.png
> Airflow takes approximately 1 second to import. It turns out that more than half the
time is spend doing two things: importing Cryptography to create Fernet keys and importing
> The first Cryptography import is in and is only necessary if Airflow
is generating a new airflow.cfg file (but currently gets run every time). Therefore it can
be easily deferred.
> The second is in to check if encryption is turned on. This can also be deferred
until encryption checks are actually needed.
> Alembic is always imported even though it is only needed for running initdb() and upgradedb().
It can be lazily imported inside those functions.
> These simple changes reduce Airflow's startup time by half on my machine.

This message was sent by Atlassian JIRA

View raw message