airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Potiuk <Jarek.Pot...@polidea.com>
Subject [PROPOSAL] Production-ready Airflow docker image and helm chart proposal
Date Mon, 07 Oct 2019 07:43:11 GMT
I just created "AIP-26 - Production-ready Airflow Docker Image and helm
chart
<https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-26+-+Production-ready+Airflow+Docker+Image+and+helm+chart>"
proposal and would love community feedback and comments.

I would love to know what is your expectation from the Docker image and how
you are using production images currently.

We were looking at the images available together with Daniel Imberman
(including puckel and Astronomer image) and I started to work on a draft POC
<https://github.com/apache/airflow/pull/6266> to incorporate building of
the production-ready image into the CI framework we have. The current POC
uses the same Dockerfile we used for CI images but run with
production-ready parameters. I made a number of refactorings and
simplifications to the build logic and the scripts should be much more sane
now (and unit-testable soon).

The images are based on Debian buster (Debian 10) LTS released few months
ago (so far we used Debian stretch for CI). It still needs some fixes for
failing tests (https://travis-ci.org/potiuk/airflow/builds/594281768) but
the image itself can be downloaded from my private repo: `docker pull
potiuk/airflow:master-python3.5`)

The Docker file is available here:
https://github.com/PolideaInternal/airflow/blob/prod-image/Dockerfile

Once we have good image we will work with Daniel on helm chart so that
Airflow can be easily installed on Kubernetes.

Any comments and feedback/discussion in the AIP-26 document are welcome !

J.

-- 

Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer

M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message