airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philippe Gagnon <philgagn...@gmail.com>
Subject Re: [Discuss] AIP-23 Proposal "Migration out of Travis CI"
Date Tue, 23 Jul 2019 19:55:25 GMT
I think that Jarek is proposing switching to Kaniko for security
considerations as GKE workloads run in OS containers (so they do not
benefit from hardware virtualization sandboxing) and docker build requires
root privileges.

In any case, Kaniko is not tied in any way to Google Cloud, so the build
infrastructure could be migrated to a new platform without problem.

On Tue, Jul 23, 2019 at 3:42 PM Shah Altaf <mendhak@gmail.com> wrote:

> Hi that's a nice writeup, easy to follow.  Also I like your diagram.
>
> Question - what is the purpose of introducing kaniko instead of using
> regular docker build?
> I'm asking in line with the consideration "The system should be
> self-maintainable - with as little special Development/Ops maintenance
> needed." ,
> Though kaniko is an open source project, if its purpose here can be done
> simply with the regular docker commands, that's fewer moving parts and a
> lower overhead for maintenance as well as the future.
>
> The reason for mentioning the future - considering how Travis has
> deteriorated and is pretty much forcing a move away, it's worth learning
> from, and any new build pipeline would benefit from being as agnostic as
> possible, with as few pieces as possible.
>
>
> Regards
> Shah
>
>
>
>
> On Tue, Jul 23, 2019 at 5:12 PM Jarek Potiuk <Jarek.Potiuk@polidea.com>
> wrote:
>
> > Hello Everyone,
> >
> > I prepared a short docs where I described general architecture of the
> > solution I imagine we can deploy fairly quickly - having GitLab CI
> support
> > and Google provided funding for GCP resources.
> >
> > I am going to start working on Proof-Of-Concept soon but before I start
> > doing it, I would like to get some comments and opinions on the proposed
> > approach. I discussed the basic approach with my friend Kamil who works
> at
> > GitLab and he is a CI maintainer and this is what we think will be
> > achievable in fairly short time.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-23+Migrate+out+of+Travis+CI
> >
> > I am happy to discuss details and make changes to the proposal - we can
> > discuss it here or as comments in the document.
> >
> > Let's see what people think about it and if we get to some consensus we
> > might want to cast a vote (or maybe go via lasy consensus as this is
> > something we should have rather quickly)
> >
> > Looking forward to your comments!
> >
> > J.
> >
> > --
> >
> > Jarek Potiuk
> > Polidea <https://www.polidea.com/> | Principal Software Engineer
> >
> > M: +48 660 796 129 <+48660796129>
> > [image: Polidea] <https://www.polidea.com/>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message