hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wellington Chevreuil <wellington.chevre...@gmail.com>
Subject Re: GridMix doesn't run ClassNotFoundException Rumen
Date Tue, 15 Dec 2015 15:27:40 GMT
Hi,

Since it's on yarn class path, it should already be on Nodemanager or the
jobs path. You can check the Nodemanager process itself, with a ps command.

Can you also check the file permissions for this jar? Make sure it's
readable by the user running Nodemanager process.
Em 15/12/2015 12:53, "siscia" <sisciamirror@yahoo.com> escreveu:

> Hi Wellington,
>
> I am missing a passage, if I check the classpath of both hadoop and yarn
> they should include hadoop-rumen-2.6.1.jar
>
> simo@simo:~/hadoop-2.6.1$ bin/hadoop classpath
>
> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:
> */home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:
> */home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**
> :/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH
>
> simo@simo:~/hadoop-2.6.1$ bin/yarn classpath
>
> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:/
> *home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:
> */home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*
> *:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*
>
>
> Given that
>
> simo@simo:~*/hadoop-2.6.1/share/hadoop/tools/lib*$ ls
> hadoop-extras-2.6.1.jar                servlet-api-2.5.jar
> *hadoop-gridmix-2.6.1.jar*               snappy-java-1.0.4.1.jar
> hadoop-openstack-2.6.1.jar             stax-api-1.0-2.jar
> hadoop-rumen-2.6.1.jar                 xmlenc-0.52.jar
> hadoop-sls-2.6.1.jar                   xz-1.0.jar
> hadoop-streaming-2.6.1.jar             zookeeper-3.4.6.jar
>
> Am I doing something wrong ? How do I check the classpath of the
> NodeManager ?
>
> Greets
>
> Simone
>
> On 15/12/2015 12:45, Wellington Chevreuil wrote:
>
> Hi Simone,
>
> You should make sure to include hadoop-rumen-2.6.0.jar on the classpath
> for the Nodemanagers, or include it on the classpath of your job.
>
> On 14 Dec 2015, at 09:56, siscia <sisciamirror@yahoo.com.INVALID
> <sisciamirror@yahoo.com.invalid>> wrote:
>
> Hello folks,
>
> I am trying to run a simulation with GridMix but the simulation fails
> every time.
>
> If I inspect the log I find the error with looks like this:
>
> "Error:
> *java.lang.ClassNotFoundException:
> org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
>
> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
> java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
> java.security.AccessController.doPrivileged(Native Method)\n\tat
> java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
> java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
> java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat
> java.lang.Class.getDeclaredConstructors0(Native Method)\n\tat
> java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat
> java.lang.Class.getConstructor0(Class.java:2895)\n\tat
> java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
> org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat
> java.security.AccessController.doPrivileged(Native Method)\n\tat
> javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>
> The error is pretty clear, but I have no idea how to fix it.
>
> I run GridMix by command line with:
>
> bin/hadoop jar share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar iopath
> trace.json
>
> Any hits ?
>
> Cheers :)
>
> Simone
>
>
>
>

Mime
View raw message