airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From manish ranjan <cse1.man...@gmail.com>
Subject Re: Airflow always dumps logs to $AIRFLOW_HOME in 1.8.2
Date Wed, 25 Oct 2017 21:54:55 GMT
Thanks for the response Daniel.
If others could not see my screenshot either, I am pasting the content

*base_log_folder = /mnt1/logs*
*child_process_log_directory = /mnt1/logs/scheduler*

Followed by a restart of airflow-scheduler and webserver (Again I did it as
a cautionary measure, may not have been required). But it continued to log
the files under * /home/hadoop/airflow/logs/scheduler/. *and I could never
get it to log to /mnt location.

Any help is appreciated.




On Wed, Oct 25, 2017 at 1:47 PM Daniel Huang <dxhuang@gmail.com> wrote:

> I can't view your screenshot, but note that there are two different configs
> for log file path in airflow.cfg. There is core:base_log_folder and
> scheduler:child_process_log_directory, the latter being the location of the
> scheduler logs.
>
> On Wed, Oct 25, 2017 at 1:44 PM, manish ranjan <cse1.manish@gmail.com>
> wrote:
>
> > The airflow version is 1.8.0. Sorry about the typo.
> >
> > Manish
> >
> > ~Manish
> >
> >
> >
> > On Wed, Oct 25, 2017 at 1:42 PM, manish ranjan <cse1.manish@gmail.com>
> > wrote:
> >
> >> Hi All,
> >>
> >> My airflow runs in production on AWS and as it is also the cluster of my
> >> Spark Cluster, I tried to redirect the logs to a separate folder
> /mnt/xyz
> >> However, I always see logs being generated under
> >> * /home/hadoop/airflow/logs/scheduler/.*
> >>
> >> I have already provided this option in airflow.cfg (See screen shot
> >> below) and restarted the webserver and scheduler (I am not sure if it
> was
> >> required, but I did it anyway).
> >>
> >>
> >>
> >> Am I doing anything wrong here ?
> >>
> >>
> >> ~Manish
> >>
> >>
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message