spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <so...@cloudera.com>
Subject Re: wholeTextFiles() : java.lang.IncompatibleClassChangeError: Found class org.apache.hadoop.mapreduce.TaskAttemptContext, but interface was expected
Date Tue, 03 Jun 2014 17:33:45 GMT
I'd try the internet / SO first -- these are actually generic
Hadoop-related issues. Here I think you don't have HADOOP_HOME or
similar set.

http://stackoverflow.com/questions/19620642/failed-to-locate-the-winutils-binary-in-the-hadoop-binary-path

On Tue, Jun 3, 2014 at 5:54 PM, toivoa <toivo.muu@gmail.com> wrote:
> Wow! What a quick reply!
>
> adding
>
>         <dependency>
>                 <groupId>org.apache.hadoop</groupId>
>                 <artifactId>hadoop-client</artifactId>
>                 <version>2.4.0</version>
>         </dependency>
>
> solved the problem.
>
> But now I get
>
> 14/06/03 19:52:50 ERROR Shell: Failed to locate the winutils binary in the
> hadoop binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in
> the Hadoop binaries.
>         at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318)
>         at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333)
>         at org.apache.hadoop.util.Shell.<clinit>(Shell.java:326)
>         at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:76)
>         at org.apache.hadoop.security.Groups.parseStaticMapping(Groups.java:93)
>         at org.apache.hadoop.security.Groups.<init>(Groups.java:77)
>         at
> org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:240)
>         at
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:255)
>         at
> org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:283)
>         at org.apache.spark.deploy.SparkHadoopUtil.<init>(SparkHadoopUtil.scala:36)
>         at
> org.apache.spark.deploy.SparkHadoopUtil$.<init>(SparkHadoopUtil.scala:109)
>         at org.apache.spark.deploy.SparkHadoopUtil$.<clinit>(SparkHadoopUtil.scala)
>
> thanks
> toivo
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/wholeTextFiles-java-lang-IncompatibleClassChangeError-Found-class-org-apache-hadoop-mapreduce-TaskAtd-tp6818p6820.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message