airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Potiuk <Jarek.Pot...@polidea.com>
Subject Re: Production images for Helm chart - question if we are doing the right thing ?
Date Wed, 10 Jun 2020 18:16:19 GMT
I hope it's only temporary measure - we are likely to release 1.10.11 in a
few weeks and then we can even delete those images. But I do agree that
adding test in the name might be a better looking one. Ash - WDYT?

On Mon, Jun 8, 2020 at 12:25 PM Zikun Zhu <zhu@zikun.dev> wrote:

> When I looked at the tag "apache/airflow:1.10.10.1-alpha1-python3.6" for
> the first time, I thought "1.10.10.1-alpha1" refers to the Airflow version,
> so I asked in Slack if this version includes a fix. I believe many (if not
> most) users do not read IMAGES.rst in detail before using the images. It
> could be better to distinguish the tag syntax from production tags
> "1.10.10*" by using something like
> "apache/airflow:helm-alpha1-1.10.10-python3.6" or
> "apache/airflow:v1-10-10-helm-test1-python3.6" (following the syntax of CI
> image tag v1-10-test-python3.6-ci).
>
> Zikun
>
> On Mon, 8 Jun 2020 at 05:13, Jarek Potiuk <Jarek.Potiuk@polidea.com>
> wrote:
>
> > I think it's more for the community to decide. The ASF has some general
> > rules that we follow, yet there is usually a lot of room for community to
> > decide.
> >
> > If we think we're good.with the rules, I believe we're good :)
> >
> > J.
> >
> > niedz., 7 cze 2020, 22:51 u┼╝ytkownik Bas Harenslak
> > <basharenslak@godatadriven.com.invalid> napisał:
> >
> > > I could give my own opinion about this thing, but is there nobody at
> > > Apache that can answer this question?
> > >
> > > Bas
> > >
> > > > On 7 Jun 2020, at 20:09, Philippe Gagnon <philgagnon1@gmail.com>
> > wrote:
> > > >
> > > > Hello Jarek,
> > > >
> > > > Since they are clearly marked as alpha releases I think this is okay.
> > > Just
> > > > my opinion though.
> > > >
> > > > Regards,
> > > >
> > > > Philippe
> > > >
> > > > On Sun, Jun 7, 2020 at 1:09 PM Jarek Potiuk <
> Jarek.Potiuk@polidea.com>
> > > > wrote:
> > > >
> > > >> Hello everyone,
> > > >>
> > > >> We have a question with Ash whether the way we are distributing
> alpha
> > > >> releases of our Docker images is ok for everyone. This is mainly to
> > the
> > > >> other PMCs in the project but anyone's opinion is welcome.
> > > >>
> > > >> *Context:*
> > > >>
> > > >> We have been iterating with the production image over the last few
> > days
> > > >> with Ash - we tried to make them usable for the helm chart and we
> had
> > to
> > > >> add a few scripts. In order to make them testable with the Helm
> chart,
> > > we
> > > >> pushed images built using official airflow 1.10.10 release but with
> a
> > > few
> > > >> modifications (some script like entrypoint and clean-logs were
> > updated).
> > > >>
> > > >> We've been thinking that this is "borderline" OK with Apache rules
> for
> > > >> unreleased software (
> > > >> https://infra.apache.org/release-distribution.html#unreleased).
> > > >> While we seem to be ok with the first three rules, the fourth one
> is a
> > > bit
> > > >> vague:
> > > >>
> > > >> * must not be distributed through channels which encourage use by
> > anyone
> > > >> outside the project development community
> > > >>
> > > >> Those images are available in the DockerHub in apache/airlfow
> project
> > > with
> > > >> names like:
> > > >>
> > > >> * apache/airflow:1.10.10.1-alpha1-python3.6
> > > >> * apache/airflow:1.10.10.1-alpha2-python3.6
> > > >>
> > > >> I have a PR where we keep track of the changes and where we explain
> > the
> > > >> reasoning behind it and clearly mark it as  "alpha/development"
> > > releases:
> > > >> https://github.com/apache/airflow/pull/9162/files
> > > >>
> > > >> *Question:*
> > > >>
> > > >> Is this OK for everyone that we follow it this way?
> > > >>
> > > >>
> > > >> *Content of the note in IMAGES.rst:*
> > > >>
> > > >> Alpha versions of 1.10.10 production-ready images
> > > >>
> > > >> The production images have been released for the first time in
> 1.10.10
> > > >> release of Airflow as "Alpha" quality ones. Between 1.10.10 the
> images
> > > are
> > > >> being improved and the 1.10.10 images should be patched and
> published
> > > >> several times separately in order to test them with the upcoming
> Helm
> > > >> Chart.
> > > >>
> > > >> Those images are for development and testing only and should not be
> > used
> > > >> outside of the development community.
> > > >>
> > > >> The images were pushed with tags following the pattern:
> > > >> apache/airflow:1.10.10.1-alphaN-pythonX.Y.. Patch level is an
> > increasing
> > > >> number (starting from 1).
> > > >>
> > > >> Those are alpha-quality releases however they contain the officially
> > > >> released Airflow 1.10.10 code. The main changes in the images are
> > > scripts
> > > >> embedded in the images.
> > > >>
> > > >> The following versions were pushed:
> > > >>
> > > >>
> > >
> >
> +-------+--------------------------------+----------------------------------------------------------+
> > > >> | Patch | Tag pattern                    | Description
> > > >>                         |
> > > >>
> > > >>
> > >
> >
> +=======+================================+==========================================================+
> > > >> | 1     | ``1.10.10.1-alpha1-pythonX.Y`` | Support for parameters
> > added
> > > to
> > > >> bash and python commands |
> > > >>
> > > >>
> > >
> >
> +-------+--------------------------------+----------------------------------------------------------+
> > > >> | 2     | ``1.10.10-1-alpha2-pythonX.Y`` | Added "/clean-logs"
> script
> > > >>                        |
> > > >>
> > > >>
> > >
> >
> +-------+--------------------------------+----------------------------------------------------------+
> > > >> The commits used to generate those images are tagged with
> > > >> prod-image-1.10.10.1-alphaN tags.
> > > >>
> > > >>
> > > >> J.
> > > >>
> > > >>
> > > >> --
> > > >>
> > > >> Jarek Potiuk
> > > >> Polidea <https://www.polidea.com/> | Principal Software Engineer
> > > >>
> > > >> M: +48 660 796 129 <+48660796129>
> > > >> [image: Polidea] <https://www.polidea.com/>
> > > >>
> > >
> > >
> >
>


-- 

Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer

M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message