hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Class org.apache.hadoop.fs.s3a.S3A not found
Date Fri, 16 Dec 2016 23:35:11 GMT
MutableCounterLong is in hadoop-common jar.

Can you check the classpath to see if there is another hadoop-common jar
(other than 3.0-alpha1) ?

Cheers

On Fri, Dec 16, 2016 at 1:49 PM, cs user <acldstkusr@gmail.com> wrote:

> Hello,
>
> Thank you both for getting back to me so quickly,
>
> Can you explain how I can get this to work with the latest version? I'm
> trying to submit a spark job with the following ( With the hadoop 3
> release).
>
> Sys.setenv('SPARKR_SUBMIT_ARGS'='"--packages" "com.amazonaws:aws-java-sdk-
> pom:1.10.34,org.apache.hadoop:hadoop-aws:3.0.0-alpha1" "sparkr-shell"',
>
> But then I get:
>
> 16/12/16 21:47:56 ERROR RBackendHandler: createSparkContext on
> org.apache.spark.api.r.RRDD failed
> Error in invokeJava(isStatic = TRUE, className, methodName, ...) :
>   java.lang.IllegalAccessError: tried to access method
> org.apache.hadoop.metrics2.lib.MutableCounterLong.<init>(
> Lorg/apache/hadoop/metrics2/MetricsInfo;J)V from class
> org.apache.hadoop.fs.s3a.S3AInstrumentation
>
>
> How can I set my environment to pull down the hadoop 2.8 release?
>
> Thank you again for your help.
>
>
>
>
>
>

Mime
View raw message