spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Graves (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-1465) Spark compilation is broken with the latest hadoop-2.4.0 release
Date Fri, 11 Apr 2014 14:19:14 GMT

    [ https://issues.apache.org/jira/browse/SPARK-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13966545#comment-13966545
] 

Thomas Graves commented on SPARK-1465:
--------------------------------------

You can't just update spark to the new api as it will break compiling against 2.3 and other
2.x versions.    I will file a bug against Hadoop for this and it should be fixed in a 2.4.1
release.  Unfortunately that doesn't help people using 2.4.  

For Spark I think we are better off just to make our own functions that do that functionality.

> Spark compilation is broken with the latest hadoop-2.4.0 release
> ----------------------------------------------------------------
>
>                 Key: SPARK-1465
>                 URL: https://issues.apache.org/jira/browse/SPARK-1465
>             Project: Spark
>          Issue Type: Bug
>            Reporter: Xuan Gong
>
> Building spark with the latest 2.4.0 version of yarn, appears to be broken
> {code}
> [ERROR]       Apps.addToEnvironment(env, Environment.CLASSPATH.name, Environment.PWD.$()
+
> [ERROR]                            ^
> [ERROR] /yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala:441:
not enough arguments for method addToEnvironment: (x$1: java.util.Map[String,String], x$2:
String, x$3: String, x$4: String)Unit.
> Unspecified value parameter x$4.
> [ERROR]     Apps.addToEnvironment(env, Environment.CLASSPATH.name, Environment.PWD.$()
+
> [ERROR]                          ^
> [ERROR] /yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala:446:
not enough arguments for method addToEnvironment: (x$1: java.util.Map[String,String], x$2:
String, x$3: String, x$4: String)Unit.
> Unspecified value parameter x$4.
> [ERROR]       Apps.addToEnvironment(env, Environment.CLASSPATH.name, Environment.PWD.$()
+
> [ERROR]                            ^
> [ERROR] /yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala:449:
not enough arguments for method addToEnvironment: (x$1: java.util.Map[String,String], x$2:
String, x$3: String, x$4: String)Unit.
> Unspecified value parameter x$4.
> [ERROR]     Apps.addToEnvironment(env, Environment.CLASSPATH.name, Environment.PWD.$()
+
> [ERROR]                          ^
> [ERROR] /yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnableUtil.scala:170:
not enough arguments for method setEnvFromInputString: (x$1: java.util.Map[String,String],
x$2: String, x$3: String)Unit.
> Unspecified value parameter x$3.
> [ERROR]     Apps.setEnvFromInputString(env, System.getenv("SPARK_YARN_USER_ENV"))
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message