hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mingjie Lai <m...@apache.org>
Subject Re: yarn NoClassDefFoundError from LinuxContainerExecutor
Date Fri, 02 Mar 2012 18:51:08 GMT

It relates to: https://issues.apache.org/jira/browse/MAPREDUCE-3505

Thanks.

On 03/01/2012 07:09 AM, Ioan Eugen Stan wrote:
> Hi Mingjie,
>
>
> I don't know about Yarn, but NoClassDefFoundError appear when you have a
> class that was present during compile time but no longer available
> during runtime. See a detailed explanation here [1].
>
> Check that the classpath built in the container/node contains the
> classes from that error. Also check that you don't get another version
> on the classpath, one without the specified class.
>
> Hope this helps,
>
>
> [1]
> http://javarevisited.blogspot.com/2011/06/noclassdeffounderror-exception-in.html
>
>
>
> Pe 29.02.2012 23:07, Mingjie Lai a scris:
>>
>> Hi.
>>
>> I'm trying yarn + security but still cannot make a mapred example
>> runing. Can anyone help me to take a look?
>>
>> My env:
>> - 3-slave cluster on ec2. Centos 5.5
>> - nn, dn, rm, nm all started, with security enabled.
>> - i saw java.lang.NoClassDefFoundError from LinuxContainerExecutor eror
>> log:
>> ./application_1330545370212_0004/container_1330545370212_0004_01_000001/stderr
>>
>>
>> - If i disable security, still saw this issue.
>>
>> Any hint?
>>
>> I followed the instructions from
>> http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html
>>
>>
>>
>> Steps:
>> I started a mapred sample from nn/rm:
>>
>> $ usr/lib/hadoop/bin/yarn --config ./conf jar
>> share/hadoop/mapreduce/hadoop-mapreduce-examples-0.24.0-SNAPSHOT.jar
>> randomwriter 10 10
>>
>> Logs are from nn, nm,
>> ------------------------------------------
>> [yarn@ip-10-176-231-35 hadoop]$ /usr/lib/hadoop/bin/yarn --config ./conf
>> jar share/hadoop/mapreduce/hadoop-mapreduce-examples-0.24.0-SNAPSHOT.jar
>> randomwriter 10 10
>> Running 30 maps.
>> Job started: Wed Feb 29 20:33:48 UTC 2012
>> 12/02/29 20:33:48 WARN conf.Configuration:
>> mapred.used.genericoptionsparser is deprecated. Instead, use
>> mapreduce.client.genericoptionsparser.used
>> 12/02/29 20:33:49 INFO mapreduce.JobSubmitter: number of splits:30
>> 12/02/29 20:33:49 INFO mapred.ResourceMgrDelegate: Submitted application
>> application_1330545370212_0005 to ResourceManager at
>> ip-10-176-231-35.us-west-1.compute.internal/10.176.231.35:7090
>> 12/02/29 20:33:49 INFO mapreduce.Job: The url to track the job:
>> http://ip-10-176-231-35.us-west-1.compute.internal:7050/proxy/application_1330545370212_0005/
>>
>>
>> 12/02/29 20:33:49 INFO mapreduce.Job: Running job: job_1330545370212_0005
>> 12/02/29 20:33:53 INFO mapreduce.Job: Job job_1330545370212_0005 running
>> in uber mode : false
>> 12/02/29 20:33:53 INFO mapreduce.Job: map 0% reduce 0%
>> 12/02/29 20:33:53 INFO mapreduce.Job: Job job_1330545370212_0005 failed
>> with state FAILED due to: Application application_1330545370212_0005
>> failed 1 times due to AM Container for
>> appattempt_1330545370212_0005_000001 exited with exitCode: 1 due to:
>> Exception from container-launch:
>> org.apache.hadoop.util.Shell$ExitCodeException:
>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
>> at org.apache.hadoop.util.Shell.run(Shell.java:188)
>> at
>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:207)
>>
>>
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:241)
>>
>>
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:68)
>>
>>
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>
>>
>> at java.lang.Thread.run(Thread.java:662)
>>
>> main : command provided 1
>> main : user is yarn
>>
>> .Failing this attempt.. Failing the application.
>> 12/02/29 20:33:53 INFO mapreduce.Job: Counters: 0
>> Job ended: Wed Feb 29 20:33:53 UTC 2012
>> The job took 5 seconds.
>> ------------------------------------------
>>
>> LinuxContainer error:
>>
>> [root@ip-10-176-203-45 yarn]# more
>> ./application_1330545370212_0004/container_1330545370212_0004_01_000001/stderr
>>
>>
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/hadoop/yar
>> n/service/CompositeService
>> at java.lang.ClassLoader.defineClass1(Native Method)
>> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
>> at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
>> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:14
>> 1)
>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
>> at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.hadoop.yarn.service.Comp
>> ositeService
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>> ... 12 more
>> Could not find the main class:
>> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.
>> Program will exit.
>> ----------------------------------------
>
>

Mime
View raw message