airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "t oo (Jira)" <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-5406) Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes]
Date Sat, 28 Dec 2019 17:05:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-5406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

t oo updated AIRFLOW-5406:
--------------------------
    Description: 
Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes]

k8s dependencies causing issues for non-k8s users based on 1.10.6:
*settings.py:*
from airflow.contrib.kubernetes.pod import Pod
def pod_mutation_hook(pod):

*models/__init__.py*
from airflow.models.kubernetes import KubeWorkerIdentifier, KubeResourceVersion

*spark_submit_hook.py*
from airflow.contrib.kubernetes import kube_client



  was:
Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes]

k8s dependencies causing issues for non-k8s users based on 1.10.6:
*settings.py:*
from airflow.contrib.kubernetes.pod import Pod
def pod_mutation_hook(pod):

*models/__init__.py*
from airflow.models.kubernetes import KubeWorkerIdentifier, KubeResourceVersion

*executors/__init__.py*
KubernetesExecutor = "KubernetesExecutor"
elif executor_name == Executors.KubernetesExecutor:
        from airflow.contrib.executors.kubernetes_executor import KubernetesExecutor
        return KubernetesExecutor()

*spark_submit_hook.py*
from airflow.contrib.kubernetes import kube_client




> Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes]
> ---------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-5406
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5406
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: dependencies, executor-kubernetes, executors
>    Affects Versions: 1.10.5
>            Reporter: t oo
>            Assignee: t oo
>            Priority: Blocker
>             Fix For: 1.10.8
>
>
> Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes]
> k8s dependencies causing issues for non-k8s users based on 1.10.6:
> *settings.py:*
> from airflow.contrib.kubernetes.pod import Pod
> def pod_mutation_hook(pod):
> *models/__init__.py*
> from airflow.models.kubernetes import KubeWorkerIdentifier, KubeResourceVersion
> *spark_submit_hook.py*
> from airflow.contrib.kubernetes import kube_client



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message