airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <>
Subject Re: Question about airflow scheduler
Date Mon, 20 Jun 2016 22:44:54 GMT
Hey Jason,

We run our scheduler without setting this. It will cause the scheduler to
kill itself ever N loops. If you use it in conjunction with a process
monitor (e.g. supervisord), you can have the scheduler kill itself, and
then the process monitor will restart it. This gives you short lived
schedulers that clear their state ever N loops.

TBH, I don't think that it's required anymore. It seems like a hack for
when the scheduler was more stateful, and needed to restart itself
periodically to clear its in-memory state.


On Sun, Jun 19, 2016 at 2:59 PM, Jason Chen <>

> Hi Airflow team,
> To run "airflow scheduler", you can specify an argument "SCHEDULER_RUNS"
> Something like "airflow scheduler -n ${SCHEDULER_RUNS}".
> What it's used for ?
> Side effects when setting it to un-limit ?
> Thanks.
> Jason

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message