airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Niels Zeilemaker (JIRA)" <>
Subject [jira] [Created] (AIRFLOW-1775) Remote file handler for logging
Date Wed, 01 Nov 2017 13:51:00 GMT
Niels Zeilemaker created AIRFLOW-1775:

             Summary: Remote file handler for logging
                 Key: AIRFLOW-1775
             Project: Apache Airflow
          Issue Type: Bug
          Components: logging
            Reporter: Niels Zeilemaker
            Priority: Normal

We're using a mounted Azure file share to store our log files. Currently, Airflow is writing
it's logs to that fileshare. However, this is making the solution quite expensive, as you
pay per action on Azure file shares.

If we would have a remote_file_task_logging handler, we could mimic the s3_task_logging, and
copy the file to the fileshare in the close method. Reducing the cost, while still providing
persistent storage.

This message was sent by Atlassian JIRA

View raw message