airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Naik Kaxil <k.n...@reply.com>
Subject Airflow - High Availability and Scale Up vs Scale Out
Date Fri, 08 Jun 2018 08:22:53 GMT
Hi guys,

I have 2 specific questions for the guys using Airflow in production?


  1.  How have you achieved High availability? How does the architecture look like? Do you
replicate the master node as well?
  2.  Scale Up vs Scale Out?
     *   What is the preferred approach you take? 1 beefy Airflow VM with Worker, Scheduler
and Webserver using Local Executor or a cluster with multiple workers using Celery Executor.

I think this thread should help others as well with similar question.


Regards,
Kaxil




Kaxil Naik

Data Reply
2nd Floor, Nova South
160 Victoria Street, Westminster
London SW1E 5LB - UK
phone: +44 (0)20 7730 6000
k.naik@reply.com<mailto:k.naik@reply.com>
www.reply.com

[Data Reply]
Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message