airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Kotecha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-2745) Use k8s service account for Kube Pod Operator if in Cluster and using LocalExecutor
Date Mon, 16 Jul 2018 16:39:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16545459#comment-16545459
] 

Ravi Kotecha commented on AIRFLOW-2745:
---------------------------------------

[~eamonkeane] maybe I don't understand what you want to do but I think setting 
{noformat}
in_cluster=True{noformat}
 when creating a KubernetesPodOperator does what you want.

> Use k8s service account for Kube Pod Operator if in Cluster and using LocalExecutor
> -----------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-2745
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2745
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: operators
>    Affects Versions: 2.0.0
>            Reporter: Eamon Keane
>            Assignee: Daniel Imberman
>            Priority: Minor
>              Labels: features
>
> When deploying airflow on kubernetes and using LocalExecutor, currently the Kubernetes
Pod Operator relies on a kubeconfig file being mounted on the scheduler and has no awareness
of being inside a cluster.
> There is no option to mount a kubeconfig file on a KubernetesExecutor worker as the KubernetesExecutor
instead launches Kubenetes Pod Operator pods using the mounted RBAC account.
> For users switching between the KubernetesExecutor and LocalExecutor in a helm chart
(for example by using --set core.executor=LocalExecutor), an additional kubeconfig secret
has to be managed and mounted on the scheduler if they want to debug a dag which uses the
Kubernetes Pod Operator, while it could instead use the RBAC account.
> An example where switching between local and kubernetes executor was useful was to discover
that the reason a dag worked on Local but not on Kubernetes Executor was because the fernet
key was not specified as an environment variable in the worker definition.
> The suggested improvement would be to use the mounted RBAC account on the scheduler pod
to launch pods if in a kubernetes environment, removing the need for a kubeconfig.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message