spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcelo Vanzin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-18840) HDFSCredentialProvider throws exception in non-HDFS security environment
Date Tue, 13 Dec 2016 18:38:59 GMT

     [ https://issues.apache.org/jira/browse/SPARK-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Marcelo Vanzin updated SPARK-18840:
-----------------------------------
    Fix Version/s: 2.2.0
                   2.1.1

> HDFSCredentialProvider throws exception in non-HDFS security environment
> ------------------------------------------------------------------------
>
>                 Key: SPARK-18840
>                 URL: https://issues.apache.org/jira/browse/SPARK-18840
>             Project: Spark
>          Issue Type: Bug
>          Components: YARN
>    Affects Versions: 1.6.3, 2.1.0
>            Reporter: Saisai Shao
>            Assignee: Saisai Shao
>            Priority: Minor
>             Fix For: 2.1.1, 2.2.0
>
>
> Current in {{HDFSCredentialProvider}}, the code logic assumes HDFS delegation token should
be existed, this is ok for HDFS environment, but for some cloud environment like Azure, HDFS
is not required, so it will throw exception:
> {code}
> java.util.NoSuchElementException: head of empty list
>         at scala.collection.immutable.Nil$.head(List.scala:337)
>         at scala.collection.immutable.Nil$.head(List.scala:334)
>         at org.apache.spark.deploy.yarn.Client.getTokenRenewalInterval(Client.scala:627)
> {code}
> We should also consider this situation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message