airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bas Harenslak <basharens...@godatadriven.com.INVALID>
Subject Re: Production images for Helm chart - question if we are doing the right thing ?
Date Sun, 07 Jun 2020 20:50:57 GMT
I could give my own opinion about this thing, but is there nobody at Apache that can answer
this question?

Bas

> On 7 Jun 2020, at 20:09, Philippe Gagnon <philgagnon1@gmail.com> wrote:
> 
> Hello Jarek,
> 
> Since they are clearly marked as alpha releases I think this is okay. Just
> my opinion though.
> 
> Regards,
> 
> Philippe
> 
> On Sun, Jun 7, 2020 at 1:09 PM Jarek Potiuk <Jarek.Potiuk@polidea.com>
> wrote:
> 
>> Hello everyone,
>> 
>> We have a question with Ash whether the way we are distributing alpha
>> releases of our Docker images is ok for everyone. This is mainly to the
>> other PMCs in the project but anyone's opinion is welcome.
>> 
>> *Context:*
>> 
>> We have been iterating with the production image over the last few days
>> with Ash - we tried to make them usable for the helm chart and we had to
>> add a few scripts. In order to make them testable with the Helm chart, we
>> pushed images built using official airflow 1.10.10 release but with a few
>> modifications (some script like entrypoint and clean-logs were updated).
>> 
>> We've been thinking that this is "borderline" OK with Apache rules for
>> unreleased software (
>> https://infra.apache.org/release-distribution.html#unreleased).
>> While we seem to be ok with the first three rules, the fourth one is a bit
>> vague:
>> 
>> * must not be distributed through channels which encourage use by anyone
>> outside the project development community
>> 
>> Those images are available in the DockerHub in apache/airlfow project with
>> names like:
>> 
>> * apache/airflow:1.10.10.1-alpha1-python3.6
>> * apache/airflow:1.10.10.1-alpha2-python3.6
>> 
>> I have a PR where we keep track of the changes and where we explain the
>> reasoning behind it and clearly mark it as  "alpha/development" releases:
>> https://github.com/apache/airflow/pull/9162/files
>> 
>> *Question:*
>> 
>> Is this OK for everyone that we follow it this way?
>> 
>> 
>> *Content of the note in IMAGES.rst:*
>> 
>> Alpha versions of 1.10.10 production-ready images
>> 
>> The production images have been released for the first time in 1.10.10
>> release of Airflow as "Alpha" quality ones. Between 1.10.10 the images are
>> being improved and the 1.10.10 images should be patched and published
>> several times separately in order to test them with the upcoming Helm
>> Chart.
>> 
>> Those images are for development and testing only and should not be used
>> outside of the development community.
>> 
>> The images were pushed with tags following the pattern:
>> apache/airflow:1.10.10.1-alphaN-pythonX.Y.. Patch level is an increasing
>> number (starting from 1).
>> 
>> Those are alpha-quality releases however they contain the officially
>> released Airflow 1.10.10 code. The main changes in the images are scripts
>> embedded in the images.
>> 
>> The following versions were pushed:
>> 
>> +-------+--------------------------------+----------------------------------------------------------+
>> | Patch | Tag pattern                    | Description
>>                         |
>> 
>> +=======+================================+==========================================================+
>> | 1     | ``1.10.10.1-alpha1-pythonX.Y`` | Support for parameters added to
>> bash and python commands |
>> 
>> +-------+--------------------------------+----------------------------------------------------------+
>> | 2     | ``1.10.10-1-alpha2-pythonX.Y`` | Added "/clean-logs" script
>>                        |
>> 
>> +-------+--------------------------------+----------------------------------------------------------+
>> The commits used to generate those images are tagged with
>> prod-image-1.10.10.1-alphaN tags.
>> 
>> 
>> J.
>> 
>> 
>> --
>> 
>> Jarek Potiuk
>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>> 
>> M: +48 660 796 129 <+48660796129>
>> [image: Polidea] <https://www.polidea.com/>
>> 


Mime
View raw message