airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dan Davydov (JIRA)" <>
Subject [jira] [Closed] (AIRFLOW-672) Allow logs to be piped into another process
Date Wed, 09 Aug 2017 19:49:00 GMT


Dan Davydov closed AIRFLOW-672.
    Resolution: Fixed

Not really relevant with the new logging abstraction anymore. We can create a piping to a
file logging handler if we want now.

> Allow logs to be piped into another process
> -------------------------------------------
>                 Key: AIRFLOW-672
>                 URL:
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: logging
>            Reporter: Dan Davydov
> Instead of writing logs to a file Airflow should be able to pipe them to a process similar
to how Apache Webserver does it: . The most
important reason for this is to allow log rotation of tasks which can get quite large on the
local worker disks. Part of this task could be moving the existing s3 log exporting logic
(and other custom logging logic) out of the airflow core.
> FWIW I have the code ready to add log rotation to Airflow itself, but this isn't as ideal
as the piping solution (separation of concerns). If there is interest in getting this merged
as a temporary measure I can do so.

This message was sent by Atlassian JIRA

View raw message