Mailing-List: contact dev-help@airflow.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airflow.incubator.apache.org
MIME-Version: 1.0
References: <CAHdP-nsJQT3STqNVzbqWJsXijHmO_7jH84TUkHDUMNRNeNG0UA@mail.gmail.com>
In-Reply-To: <CAHdP-nsJQT3STqNVzbqWJsXijHmO_7jH84TUkHDUMNRNeNG0UA@mail.gmail.com>
From: Sergei Iakhnin <llevar@gmail.com>
Date: Thu, 01 Sep 2016 14:54:10 +0000
Message-ID: <CAE1CdVLFq7pieEkokmxBvUoOa2TK8G6w6S40oeRtJ6-OB6Vafg@mail.gmail.com>
Subject: Re: Issue with Airflow - Jobs are not triggered to run after a random
 period of time
To: dev@airflow.incubator.apache.org
Content-Type: multipart/alternative; boundary=001a114e1612987f15053b736332
archived-at: Thu, 01 Sep 2016 14:54:28 -0000

--001a114e1612987f15053b736332
Content-Type: text/plain; charset=UTF-8

As far as I know even Airbnb themselves restart their schedulers every 30
minutes because of this issue. I ended up doing it as well with a cron job
after giving up hope that it would be fixed in the short term.

On Thu, 1 Sep 2016, 16:03 Charalampos Paravalos, <babis@rais.io> wrote:

> Hi,
>
> I am writting to ask for advise in an issue that I have with airflow and
> til now I have not managed to resolve. Wondering if someone else had
> something similar in the past.
>
> So, we use airflow to schedule DAGs that will run some jobs periodically
> (every 30min/1hr). Jobs run as normal etc., but there are some times that
> suddenly after DAGs are finished, the next scheduled jobs do not start at
> all. It seems like the server does not kick off the scheduled jobs at all,
> for any of the DAGs defined (so no jobs are running on our server). When
> that happens I have to restart the scheduler so jobs are kicked on
> automatically after restart. And the jobs run until this issue appears
> again (I noticed it happening every 1 or 2 days, it is quite often).
>
> This is very strange, tried to upgrade to 1.7.1.3 version but still that
> issue is here. We use 32 concurrent jobs with celery workers, the server is
> able to manage the load well.
>
> I believe it has to do with the scheduler, but can't understand why.
> Backfilled jobs maybe? Can this be?
>
> I am looking forward to hearing back from someone that has any ideas.
> Please let me know what information you might need about my setup anytime.
>
> Thanks for your help!
>
> Regards,
> Babis
>
-- 

Sergei

--001a114e1612987f15053b736332--