hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From siscia <sisciamir...@yahoo.com.INVALID>
Subject Re: GridMix doesn't run ClassNotFoundException Rumen
Date Wed, 16 Dec 2015 16:28:04 GMT
Hi :)

It seems perfectly fine:

    simo@simo:~/hadoop-2.6.1$ jar -tvf
    share/hadoop/tools/lib/hadoop-rumen-2.6.1.jar | grep
    ResourceUsageMetrics
       3696 Wed Sep 16 21:17:24 CEST 2015
    org/apache/hadoop/tools/rumen/ResourceUsageMetrics.class


I honestly believe that the problem is something way more trivial...

But I have no idea what to check out...

Cheers

Simone

On 16/12/2015 13:20, Wellington Chevreuil wrote:
> Hi,
>
> It seems the classpath definition is ok. Can you inspect the rumen jar 
> to make sure the file is not corrupt somehow?
>
> jar -tvf share/hadoop/tools/lib/hadoop-rumen-2.6.1.jar | grep 
> ResourceUsageMetrics
>
>> On 15 Dec 2015, at 17:14, siscia <sisciamirror@yahoo.com 
>> <mailto:sisciamirror@yahoo.com>> wrote:
>>
>> Hi,
>>
>> It looks like everything is in place but still doesn't work...
>>
>> Let me recap, I am running in pseudo distributed mode.
>>
>> I start the system with this sequence of command.
>>
>>     cd hadoop-2.6.1
>>     sbin/start-dfs.sh
>>     sbin/start-yarn.sh
>>     sbin/hadoop-daemon.sh start datanode
>>     sbin/yarn-daemon.sh start nodemanager
>>
>>
>> Now the single node of the system is healthy and everything looks cool.
>>
>> jps give me this:
>>
>>     simo@simo:~/hadoop-2.6.1$ jps
>>     10107 Jps
>>     4114 NodeManager
>>     3533 SecondaryNameNode
>>     3344 NameNode
>>     4014 DataNode
>>     3737 ResourceManager
>>
>>
>> I check which user is running the NodeManager
>>
>>     simo@simo:~/hadoop-2.6.1$ ps u 4114
>>     USER       PID %CPU %MEM    VSZ RSS TTY      STAT START   TIME
>>     COMMAND
>>     simo      4114  1.9  8.7 1764708 346188 pts/3  Sl   17:37   0:39
>>     /usr/bin/java -Dproc_nodemanager -Xmx1000m -server
>>     -Dhadoop.log.dir=/home/simo/hadoop
>>
>>
>> I am running the nodemanager.
>>
>> Let me check the permission of the rumen jar
>>
>>     simo@simo:~/hadoop-2.6.1$ ls -l share/hadoop/tools/lib/ | grep rumen
>>     -rw-r--r-- 1 simo simo   278018 set 16 23:18 hadoop-rumen-2.6.1.jar
>>
>>
>> I can read the rumen jar.
>>
>> I don't have a lot of ideas of what I could be doing wrong...
>>
>> You have any thoughts ?
>>
>> Thanks for your time,
>>
>> Simone
>>
>>
>> On 15/12/2015 16:27, Wellington Chevreuil wrote:
>>>
>>> Hi,
>>>
>>> Since it's on yarn class path, it should already be on Nodemanager 
>>> or the jobs path. You can check the Nodemanager process itself, with 
>>> a ps command.
>>>
>>> Can you also check the file permissions for this jar? Make sure it's 
>>> readable by the user running Nodemanager process.
>>>
>>> Em 15/12/2015 12:53, "siscia" <sisciamirror@yahoo.com> escreveu:
>>>
>>>     Hi Wellington,
>>>
>>>     I am missing a passage, if I check the classpath of both hadoop
>>>     and yarn they should include hadoop-rumen-2.6.1.jar
>>>
>>>         simo@simo:~/hadoop-2.6.1$
>>>         <mailto:simo@simo:%7E/hadoop-2.6.1$> bin/hadoop classpath
>>>         /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH
>>>
>>>         simo@simo:~/hadoop-2.6.1$
>>>         <mailto:simo@simo:%7E/hadoop-2.6.1$> bin/yarn classpath
>>>         /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:/*home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*
>>>
>>>
>>>     Given that
>>>
>>>         simo@simo:~*/hadoop-2.6.1/share/hadoop/tools/lib*$ ls
>>>         hadoop-extras-2.6.1.jar servlet-api-2.5.jar
>>>         *hadoop-gridmix-2.6.1.jar* snappy-java-1.0.4.1.jar
>>>         hadoop-openstack-2.6.1.jar stax-api-1.0-2.jar
>>>         hadoop-rumen-2.6.1.jar xmlenc-0.52.jar
>>>         hadoop-sls-2.6.1.jar xz-1.0.jar
>>>         hadoop-streaming-2.6.1.jar zookeeper-3.4.6.jar
>>>
>>>     Am I doing something wrong ? How do I check the classpath of the
>>>     NodeManager ?
>>>
>>>     Greets
>>>
>>>     Simone
>>>
>>>     On 15/12/2015 12:45, Wellington Chevreuil wrote:
>>>>     Hi Simone,
>>>>
>>>>     You should make sure to include hadoop-rumen-2.6.0.jar on the
>>>>     classpath for the Nodemanagers, or include it on the classpath
>>>>     of your job.
>>>>
>>>>>     On 14 Dec 2015, at 09:56, siscia
>>>>>     <sisciamirror@yahoo.com.INVALID> wrote:
>>>>>
>>>>>     Hello folks,
>>>>>
>>>>>     I am trying to run a simulation with GridMix but the
>>>>>     simulation fails every time.
>>>>>
>>>>>     If I inspect the log I find the error with looks like this:
>>>>>
>>>>>         "Error:
>>>>>         *java.lang.ClassNotFoundException:
>>>>>         org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
>>>>>
>>>>>         java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
>>>>>         java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
>>>>>         java.security.AccessController.doPrivileged(Native
>>>>>         Method)\n\tat
>>>>>         java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
>>>>>         java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
>>>>>         java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat java.lang.Class.getDeclaredConstructors0(Native
>>>>>         Method)\n\tat
>>>>>         java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat
>>>>>         java.lang.Class.getConstructor0(Class.java:2895)\n\tat
>>>>>         java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat
>>>>>         org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
>>>>>         org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
>>>>>         org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
>>>>>         org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
>>>>>         org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat
>>>>>         org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
>>>>>         org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat
>>>>>         java.security.AccessController.doPrivileged(Native
>>>>>         Method)\n\tat
>>>>>         javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
>>>>>         org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
>>>>>         org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>>>>>
>>>>>     The error is pretty clear, but I have no idea how to fix it.
>>>>>
>>>>>     I run GridMix by command line with:
>>>>>
>>>>>         bin/hadoop jar
>>>>>         share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar iopath
>>>>>         trace.json
>>>>>
>>>>>     Any hits ?
>>>>>
>>>>>     Cheers :)
>>>>>
>>>>>     Simone
>>>>
>>>
>>
>


Mime
View raw message