airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Lam <ke...@fathomhealth.co>
Subject Airflow Crashing after Azure AgentPools are re-sized
Date Tue, 24 Oct 2017 17:16:15 GMT
Hi,

We are running airflow on kubernetes, hosted on Azure, using
https://github.com/mumoshu/kube-airflow. We are also running an autoscaler:
https://github.com/wbuchwalter/Kubernetes-acs-engine-autoscaler.

Whenever the autoscaler increase the agentPool sizes to allocate a new node
in kubernetes, airflow will crash.

It looks like the worker pod crashes causing the webserver to also quit.

I don't have any logs handy at the moment, but I can try and get some.

Is anyone running a similar set-up or has anyone encountered something like
this? Appreciate any ideas.

Thanks for your help!

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message