airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edgardo Vega <edgardo.v...@gmail.com>
Subject Re: LocalExecutor restarts
Date Wed, 08 Mar 2017 01:35:13 GMT
I will try to switch over that tomorrow. Thanks for the recommendation.

Cheers,

Edgardo

On Tue, Mar 7, 2017 at 10:44 AM, Maxime Beauchemin <
maximebeauchemin@gmail.com> wrote:

> Personally I'd recommend using the CeleryExecutor (`airflow worker`) on the
> same machine as opposed to the LocalExecutor which runs as a subprocess in
> the scheduler process. Note that there's anything wrong with LocalExecutor
> except for the fact that you cannot start/stop the processes independently.
>
> Max
>
> On Tue, Mar 7, 2017 at 4:59 AM, Edgardo Vega <edgardo.vega@gmail.com>
> wrote:
>
> > I have read that you are supposed to restart the scheduler after 5-10 or
> so
> > runs. Is that still recommended? I saw the following on the google groups
> > https://groups.google.com/d/topic/airbnb_airflow/96hd61T7kgg/discussion
> > but
> > could not tell what the status was. I am currently running the latest
> > 1.7.13 installed via pip.
> >
> > I am replacing a convoluted mess of crons with airflow and started with
> the
> > LocalExecutor to execute what turns out to be a few dags at the end of
> > every hour.
> >
> >
> > --
> > Cheers,
> >
> > Edgardo
> >
>



-- 
Cheers,

Edgardo

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message