airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxime Beauchemin <maximebeauche...@gmail.com>
Subject Re: RFC: Managing task credentials inside KubernetesExecutor
Date Mon, 11 Sep 2017 16:16:58 GMT
Hi,

The proposal seems rational to me. `BaseOperator.executor_config` seems
like a good [new] place to put this. I'd assume that in some environments
there would be rules in the policy function
<https://airflow.incubator.apache.org/concepts.html#cluster-policy> to
force values in certain/all contexts.

Max

On Thu, Aug 31, 2017 at 10:17 PM, Feng Lu <fenglu@google.com.invalid> wrote:

> Sounds great, thanks a lot for setting up the meeting and will be there.
>
> On Thu, Aug 31, 2017 at 4:10 PM, Daniel Imberman <
> daniel.imberman@gmail.com>
> wrote:
>
> > Thank you for posting this to the wiki Feng Lu :).
> >
> > I'm going to propose an overall "airflow + kubernetes update" meeting in
> a
> > seperate email to discuss with the community at large. Would love it if
> you
> > could discuss this further at that meeting!
> >
> > Daniel
> >
> > On Wed, Aug 30, 2017 at 10:38 PM Feng Lu <fenglu@google.com.invalid>
> > wrote:
> >
> > > Hi all,
> > >
> > > *TL;DR*
> > > Airflow doesn't have adequate built-in support for managing per-task
> > > credentials, the concept of connection helps to certain extent but is
> not
> > > very satisfactory. The current Airflow KubernetesExecutor work opens up
> > the
> > > possibility to handle task credentials at the framework level and
> > separate
> > > workflow business logic from credential/account management by
> leveraging
> > > the Kubernetes initializer mechanism. At the end of the day, a task/dag
> > > only needs to specify an account name and everything else is taken care
> > by
> > > the Airflow framework in a secure fashion.
> > >
> > > Detailed design:
> > >
> > > https://cwiki.apache.org/confluence/display/AIRFLOW/
> > Managing+Per-task+Credentials+in+KubernetesExecutor
> > >
> > > Critics and comments are welcome :-)
> > > Thank you.
> > >
> > > Feng
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message