hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Morbious <knowledgefromgro...@gmail.com>
Subject Re: Yarn setup problems
Date Fri, 20 Mar 2015 07:17:39 GMT
Hi,

In container logs I have :

cat
 /srv/data/*/yarn/logs/application_1426770785980_0010/container_e61_1426770785980_0010_01_0000*/stderr
Error: Could not find or load main class 8345
Error: Could not find or load main class 8345
Error: Could not find or load main class 8345
Error: Could not find or load main class 8345
Error: Could not find or load main class 8345
Error: Could not find or load main class 8345
Error: Could not find or load main class 8345
Error: Could not find or load main class 8345
Error: Could not find or load main class 8345
Error: Could not find or load main class 8345
Error: Could not find or load main class 8345

In command line I get:

Job Finished in 22.395 seconds
java.io.FileNotFoundException: File does not exist:
hdfs://spark-cluster-nn/user/morbious/QuasiMonteCarlo_1426835737919_2089306463/out/reduce-out
at
org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1093)
at
org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1085)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1085)
at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1749)
at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1773)
at
org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:314)
at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:354)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:363)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:145)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

Best regards,

Morbious

2015-03-20 0:32 GMT+01:00 Jeff Zhang <zjffdu@gmail.com>:

> Please check the app logs.
>
> On Fri, Mar 20, 2015 at 5:46 AM, Morbious <knowledgefromgroups@gmail.com>
> wrote:
>
>> Hello,
>>
>> Recently I switched from MR1 to YARN on my cluster.
>> When I tried some tests like :
>>
>> > hadoop jar
>> /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.5.0-cdh5.3.2.jar pi
>> 16 10
>>
>> I got errors like:
>>
>> 15/03/19 21:40:03 INFO mapreduce.Job: Task Id :
>> attempt_1426770785980_0012_m_000006_2, Status : FAILED
>> Exception from container-launch.
>> Container id: container_e61_1426770785980_0012_01_000057
>> Exit code: 1
>> Stack trace: ExitCodeException exitCode=1:
>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>> at org.apache.hadoop.util.Shell.run(Shell.java:455)
>> at
>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:197)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:745)
>>
>>
>> Container exited with a non-zero exit code 1
>>
>> --------------------------------------------------------------------------
>>
>> In nodemanager log there is something like this:
>>
>> 2015-03-19 21:40:00,957 WARN
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exit
>> code from container container_e61_1426770785980_0012_01_000057 is : 1
>> 2015-03-19 21:40:00,957 WARN
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor:
>> Exception from container-launch with container ID:
>> container_e61_1426770785980_0012_01_000057 and exit code: 1
>> ExitCodeException exitCode=1:
>>         at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>>         at org.apache.hadoop.util.Shell.run(Shell.java:455)
>>         at
>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
>>         at
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:197)
>>         at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
>>         at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>         at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>         at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>         at java.lang.Thread.run(Thread.java:745)
>> 2015-03-19 21:40:00,958 INFO
>> org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Exception from
>> container-launch.
>> 2015-03-19 21:40:00,958 INFO
>> org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Container id:
>> container_e61_1426770785980_0012_01_000057
>> 2015-03-19 21:40:00,958 INFO
>> org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Exit code: 1
>> 2015-03-19 21:40:00,958 INFO
>> org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Stack trace:
>> ExitCodeException exitCode=1:
>> 2015-03-19 21:40:00,958 INFO
>> org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:       at
>> org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>> 2015-03-19 21:40:00,958 INFO
>> org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:       at
>> org.apache.hadoop.util.Shell.run(Shell.java:455)
>> 2015-03-19 21:40:00,958 INFO
>> org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:       at
>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
>> 2015-03-19 21:40:00,958 INFO
>> org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:       at
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:197)
>> 2015-03-19 21:40:00,958 INFO
>> org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:       at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
>> 2015-03-19 21:40:00,958 INFO
>> org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:       at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>> 2015-03-19 21:40:00,958 INFO
>> org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:       at
>> java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> 2015-03-19 21:40:00,958 INFO
>> org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:       at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> 2015-03-19 21:40:00,958 INFO
>> org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:       at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> 2015-03-19 21:40:00,958 INFO
>> org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:       at
>> java.lang.Thread.run(Thread.java:745)
>>
>>
>> I was trying to find out what went wrong, but couldn't find anything. I
>> have similiar cluster with the same configuration which runs YARN too
>> without any problems...
>>
>> Best regards,
>>
>> Morbious
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Mime
View raw message