spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hyukjin Kwon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-24905) Spark 2.3 Internal URL env variable
Date Thu, 26 Jul 2018 02:55:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-24905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556817#comment-16556817
] 

Hyukjin Kwon commented on SPARK-24905:
--------------------------------------

(please avoid to set Critical+ which is usually reserved for committers)

> Spark 2.3 Internal URL env variable
> -----------------------------------
>
>                 Key: SPARK-24905
>                 URL: https://issues.apache.org/jira/browse/SPARK-24905
>             Project: Spark
>          Issue Type: Bug
>          Components: Kubernetes
>    Affects Versions: 2.3.1
>            Reporter: Björn Wenzel
>            Priority: Major
>
> Currently the Kubernetes Master internal URL is hardcoded in the Constants.scala file
([https://github.com/apache/spark/blob/85fe1297e35bcff9cf86bd53fee615e140ee5bfb/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Constants.scala#L75)]
> In some cases these URL should be changed e.g. if the Certificate is valid for another
Hostname.
> Is it possible to make this URL a property like: spark.kubernetes.authenticate.driver.hostname?
> Kubernetes The Hard Way maintained by Kelsey Hightower for example uses kubernetes.default
as hostname, this will produce again a SSLPeerUnverifiedException.
>  
> Here is the use of the Hardcoded Host: [https://github.com/apache/spark/blob/85fe1297e35bcff9cf86bd53fee615e140ee5bfb/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala#L52]
maybe this could be changed like the KUBERNETES_NAMESPACE property in Line 53.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message