hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ioan Eugen Stan <stan.ieu...@gmail.com>
Subject Re: yarn NoClassDefFoundError from LinuxContainerExecutor
Date Thu, 01 Mar 2012 15:09:50 GMT
Hi Mingjie,


I don't know about Yarn, but NoClassDefFoundError appear when you have a 
class that was present during compile time but no longer available 
during runtime. See a detailed explanation here [1].

Check that the classpath built in the container/node contains the 
classes from that error. Also check that you don't get another version 
on the classpath, one without the specified class.

Hope this helps,


[1] 
http://javarevisited.blogspot.com/2011/06/noclassdeffounderror-exception-in.html


Pe 29.02.2012 23:07, Mingjie Lai a scris:
>
> Hi.
>
> I'm trying yarn + security but still cannot make a mapred example
> runing. Can anyone help me to take a look?
>
> My env:
> - 3-slave cluster on ec2. Centos 5.5
> - nn, dn, rm, nm all started, with security enabled.
> - i saw java.lang.NoClassDefFoundError from LinuxContainerExecutor eror
> log:
> ./application_1330545370212_0004/container_1330545370212_0004_01_000001/stderr
>
> - If i disable security, still saw this issue.
>
> Any hint?
>
> I followed the instructions from
> http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html
>
>
> Steps:
> I started a mapred sample from nn/rm:
>
> $ usr/lib/hadoop/bin/yarn --config ./conf jar
> share/hadoop/mapreduce/hadoop-mapreduce-examples-0.24.0-SNAPSHOT.jar
> randomwriter 10 10
>
> Logs are from nn, nm,
> ------------------------------------------
> [yarn@ip-10-176-231-35 hadoop]$ /usr/lib/hadoop/bin/yarn --config ./conf
> jar share/hadoop/mapreduce/hadoop-mapreduce-examples-0.24.0-SNAPSHOT.jar
> randomwriter 10 10
> Running 30 maps.
> Job started: Wed Feb 29 20:33:48 UTC 2012
> 12/02/29 20:33:48 WARN conf.Configuration:
> mapred.used.genericoptionsparser is deprecated. Instead, use
> mapreduce.client.genericoptionsparser.used
> 12/02/29 20:33:49 INFO mapreduce.JobSubmitter: number of splits:30
> 12/02/29 20:33:49 INFO mapred.ResourceMgrDelegate: Submitted application
> application_1330545370212_0005 to ResourceManager at
> ip-10-176-231-35.us-west-1.compute.internal/10.176.231.35:7090
> 12/02/29 20:33:49 INFO mapreduce.Job: The url to track the job:
> http://ip-10-176-231-35.us-west-1.compute.internal:7050/proxy/application_1330545370212_0005/
>
> 12/02/29 20:33:49 INFO mapreduce.Job: Running job: job_1330545370212_0005
> 12/02/29 20:33:53 INFO mapreduce.Job: Job job_1330545370212_0005 running
> in uber mode : false
> 12/02/29 20:33:53 INFO mapreduce.Job: map 0% reduce 0%
> 12/02/29 20:33:53 INFO mapreduce.Job: Job job_1330545370212_0005 failed
> with state FAILED due to: Application application_1330545370212_0005
> failed 1 times due to AM Container for
> appattempt_1330545370212_0005_000001 exited with exitCode: 1 due to:
> Exception from container-launch:
> org.apache.hadoop.util.Shell$ExitCodeException:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at
> org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:207)
>
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:241)
>
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:68)
>
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
> at java.lang.Thread.run(Thread.java:662)
>
> main : command provided 1
> main : user is yarn
>
> .Failing this attempt.. Failing the application.
> 12/02/29 20:33:53 INFO mapreduce.Job: Counters: 0
> Job ended: Wed Feb 29 20:33:53 UTC 2012
> The job took 5 seconds.
> ------------------------------------------
>
> LinuxContainer error:
>
> [root@ip-10-176-203-45 yarn]# more
> ./application_1330545370212_0004/container_1330545370212_0004_01_000001/stderr
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/yar
> n/service/CompositeService
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:14
> 1)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.service.Comp
> ositeService
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> ... 12 more
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.
> Program will exit.
> ----------------------------------------


-- 
Ioan Eugen Stan
http://ieugen.blogspot.com

Mime
View raw message