Mailing-List: contact dev-help@airflow.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airflow.incubator.apache.org
MIME-Version: 1.0
References: <CAG0BdMUERtAw8OCeEYtV7dGk3jgnkoNR5_D4p11etnCfv7=pMg@mail.gmail.com>
 <CAG0BdMULpBDfcgxSEbWxca=vUmLYakDGc2CB7wgRb9W51MSGtQ@mail.gmail.com> <CABDm2Q-_mCWJtYHLQAg6N7CCj+OA5AwCD4DNwZnSVeQzZVGz=Q@mail.gmail.com>
In-Reply-To: <CABDm2Q-_mCWJtYHLQAg6N7CCj+OA5AwCD4DNwZnSVeQzZVGz=Q@mail.gmail.com>
From: manish ranjan <cse1.manish@gmail.com>
Date: Wed, 25 Oct 2017 21:54:55 +0000
Message-ID: <CAG0BdMVKcNzzWo7BBZSUjLzQhU9_GYgkmF-F-KwvjWD5gRP5wA@mail.gmail.com>
Subject: Re: Airflow always dumps logs to $AIRFLOW_HOME in 1.8.2
To: dev@airflow.incubator.apache.org
Content-Type: multipart/alternative; boundary="001a1149518cbf1713055c661b84"
archived-at: Wed, 25 Oct 2017 21:55:22 -0000

--001a1149518cbf1713055c661b84
Content-Type: text/plain; charset="UTF-8"

Thanks for the response Daniel.
If others could not see my screenshot either, I am pasting the content

*base_log_folder = /mnt1/logs*
*child_process_log_directory = /mnt1/logs/scheduler*

Followed by a restart of airflow-scheduler and webserver (Again I did it as
a cautionary measure, may not have been required). But it continued to log
the files under * /home/hadoop/airflow/logs/scheduler/. *and I could never
get it to log to /mnt location.

Any help is appreciated.


On Wed, Oct 25, 2017 at 1:47 PM Daniel Huang <dxhuang@gmail.com> wrote:

> I can't view your screenshot, but note that there are two different configs
> for log file path in airflow.cfg. There is core:base_log_folder and
> scheduler:child_process_log_directory, the latter being the location of the
> scheduler logs.
>
> On Wed, Oct 25, 2017 at 1:44 PM, manish ranjan <cse1.manish@gmail.com>
> wrote:
>
> > The airflow version is 1.8.0. Sorry about the typo.
> >
> > Manish
> >
> > ~Manish
> >
> >
> >
> > On Wed, Oct 25, 2017 at 1:42 PM, manish ranjan <cse1.manish@gmail.com>
> > wrote:
> >
> >> Hi All,
> >>
> >> My airflow runs in production on AWS and as it is also the cluster of my
> >> Spark Cluster, I tried to redirect the logs to a separate folder
> /mnt/xyz
> >> However, I always see logs being generated under
> >> * /home/hadoop/airflow/logs/scheduler/.*
> >>
> >> I have already provided this option in airflow.cfg (See screen shot
> >> below) and restarted the webserver and scheduler (I am not sure if it
> was
> >> required, but I did it anyway).
> >>
> >>
> >>
> >> Am I doing anything wrong here ?
> >>
> >>
> >> ~Manish
> >>
> >>
> >>
> >
>

--001a1149518cbf1713055c661b84--