airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charalampos Paravalos <ba...@rais.io>
Subject Issue with Airflow - Jobs are not triggered to run after a random period of time
Date Thu, 01 Sep 2016 12:09:22 GMT
Hi,

I am writting to ask for advise in an issue that I have with airflow and
til now I have not managed to resolve. Wondering if someone else had
something similar in the past.

So, we use airflow to schedule DAGs that will run some jobs periodically
(every 30min/1hr). Jobs run as normal etc., but there are some times that
suddenly after DAGs are finished, the next scheduled jobs do not start at
all. It seems like the server does not kick off the scheduled jobs at all,
for any of the DAGs defined (so no jobs are running on our server). When
that happens I have to restart the scheduler so jobs are kicked on
automatically after restart. And the jobs run until this issue appears
again (I noticed it happening every 1 or 2 days, it is quite often).

This is very strange, tried to upgrade to 1.7.1.3 version but still that
issue is here. We use 32 concurrent jobs with celery workers, the server is
able to manage the load well.

I believe it has to do with the scheduler, but can't understand why.
Backfilled jobs maybe? Can this be?

I am looking forward to hearing back from someone that has any ideas.
Please let me know what information you might need about my setup anytime.

Thanks for your help!

Regards,
Babis

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message