giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From D Adams <dadam...@gmail.com>
Subject Re: Help with Giraph on Yarn
Date Thu, 20 Nov 2014 01:17:46 GMT
Hi Allessandro,
     I made a copy of the giraph examples in my .../hadoop/share folder.
This is more hopeful, but it is stuck in an infinite loop. I will try to do
some searching regarding this ...
Running hduser script ...
bin/hadoop jar
/usr/local/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
-vip /user/hduser/input/tiny_graph.txt -vof
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/user/hduser/output/shortestpaths -w 1 -yj
giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
14/11/17 12:49:11 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
14/11/17 12:49:14 INFO utils.ConfigurationUtils: No edge input format
specified. Ensure your InputFormat does not require one.
14/11/17 12:49:14 INFO utils.ConfigurationUtils: No edge output format
specified. Ensure your OutputFormat does not require one.
14/11/17 12:49:14 INFO yarn.GiraphYarnClient: Final output path is:
hdfs://localhost:9000/user/hduser/output/shortestpaths
14/11/17 12:49:14 INFO yarn.GiraphYarnClient: Running Client
14/11/17 12:49:14 INFO client.RMProxy: Connecting to ResourceManager at
localhost/127.0.0.1:8050
14/11/17 12:49:15 INFO yarn.GiraphYarnClient: Got node report from ASM for,
nodeId=roosevelt:34304, nodeAddress roosevelt:8042, nodeRackName
/default-rack, nodeNumContainers 0
14/11/17 12:49:15 INFO yarn.GiraphYarnClient: Defaulting per-task heap size
to 1024MB.
14/11/17 12:49:15 INFO yarn.GiraphYarnClient: Obtained new Application ID:
application_1416227900862_0012
14/11/17 12:49:15 INFO Configuration.deprecation: mapred.job.id is
deprecated. Instead, use mapreduce.job.id
14/11/17 12:49:15 INFO yarn.GiraphYarnClient: Set the environment for the
application master
14/11/17 12:49:15 INFO yarn.GiraphYarnClient: Environment for AM
:{CLASSPATH=${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*}
14/11/17 12:49:15 INFO yarn.GiraphYarnClient: buildLocalResourceMap ....
14/11/17 12:49:16 INFO yarn.YarnUtils: Registered file in LocalResources ::
hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application_1416227900862_0012/giraph-conf.xml
14/11/17 12:49:16 INFO yarn.GiraphYarnClient: LIB JARS
:giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
14/11/17 12:49:16 INFO yarn.YarnUtils: Class path name .
14/11/17 12:49:16 INFO yarn.YarnUtils: base path checking .
14/11/17 12:49:16 INFO yarn.GiraphYarnClient: Made local resource for
:/usr/local/hadoop/share/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
to
hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application_1416227900862_0012/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
14/11/17 12:49:20 INFO yarn.YarnUtils: Registered file in LocalResources ::
hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application_1416227900862_0012/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
14/11/17 12:49:20 INFO yarn.GiraphYarnClient: ApplicationSumbissionContext
for GiraphApplicationMaster launch container is populated.
14/11/17 12:49:20 INFO yarn.GiraphYarnClient: Submitting application to ASM
14/11/17 12:49:20 INFO impl.YarnClientImpl: Submitted application
application_1416227900862_0012
14/11/17 12:49:20 INFO yarn.GiraphYarnClient: Got new appId after
submission :application_1416227900862_0012
14/11/17 12:49:20 INFO yarn.GiraphYarnClient: GiraphApplicationMaster
container request was submitted to ResourceManager for job: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation
14/11/17 12:49:21 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 0.85
secs
14/11/17 12:49:21 INFO yarn.GiraphYarnClient:
appattempt_1416227900862_0012_000001, State: ACCEPTED, Containers used: 1
14/11/17 12:49:25 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 4.99
secs
14/11/17 12:49:25 INFO yarn.GiraphYarnClient:
appattempt_1416227900862_0012_000001, State: ACCEPTED, Containers used: 1
14/11/17 12:49:29 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 9.01
secs
14/11/17 12:49:29 INFO yarn.GiraphYarnClient:
appattempt_1416227900862_0012_000001, State: RUNNING, Containers used: 1
14/11/17 12:49:33 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 13.14
secs
14/11/17 12:49:33 INFO yarn.GiraphYarnClient:
appattempt_1416227900862_0012_000001, State: RUNNING, Containers used: 3
14/11/17 12:49:37 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 17.37
secs
14/11/17 12:49:37 INFO yarn.GiraphYarnClient:
appattempt_1416227900862_0012_000001, State: RUNNING, Containers used: 3
14/11/17 12:49:41 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 21.59
secs
14/11/17 12:49:41 INFO yarn.GiraphYarnClient:
appattempt_1416227900862_0012_000001, State: RUNNING, Containers used: 3
14/11/17 12:49:45 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 25.69
secs
14/11/17 12:49:45 INFO yarn.GiraphYarnClient:
appattempt_1416227900862_0012_000001, State: RUNNING, Containers used: 3
14/11/17 12:49:49 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 29.70
secs
14/11/17 12:49:49 INFO yarn.GiraphYarnClient:
appattempt_1416227900862_0012_000001, State: RUNNING, Containers used: 3
14/11/17 12:49:53 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 33.71
secs
14/11/17 12:49:53 INFO yarn.GiraphYarnClient:
appattempt_1416227900862_0012_000001, State: RUNNING, Containers used: 3
14/11/17 12:49:57 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 37.72
secs
14/11/17 12:49:57 INFO yarn.GiraphYarnClient:
appattempt_1416227900862_0012_000001, State: RUNNING, Containers used: 3
14/11/17 12:50:01 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 41.72
secs
14/11/17 12:50:01 INFO yarn.GiraphYarnClient:
appattempt_1416227900862_0012_000001, State: RUNNING, Containers used: 3
14/11/17 12:50:05 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 45.73
secs
14/11/17 12:50:05 INFO yarn.GiraphYarnClient:
appattempt_1416227900862_0012_000001, State: RUNNING, Containers used: 3
14/11/17 12:50:09 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 49.74
secs
14/11/17 12:50:09 INFO yarn.GiraphYarnClient:
appattempt_1416227900862_0012_000001, State: RUNNING, Containers used: 3
14/11/17 12:50:13 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 53.75
secs
...




On Wed, Nov 19, 2014 at 5:04 PM, Alessandro Negro <alenegro81@yahoo.it>
wrote:

> Could you try to copy this file:
>
> giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,
> giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>
>
> into hadoop share library (in my case /opt/yarn/hadoop-2.5.1/share/myLib)
>
> let me know
>
>
> Il giorno 19/nov/2014, alle ore 23:16, D Adams <dadamszx@gmail.com> ha
> scritto:
>
> Hi Alessandro,
>      You were expecting Doug's response, but he is my output in the
> meanwhile:
>
> hduser@Roosevelt:/usr/local/hadoop$ ./hduser_jobs.sh
> Running hduser script ...
> bin/hadoop jar
> /usr/local/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
> -vip /user/hduser/input/tiny_graph.txt -vof
> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
> /user/hduser/output/shortestpaths -w 1 -yj
> giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
> 14/11/17 09:57:45 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 14/11/17 09:57:47 INFO utils.ConfigurationUtils: No edge input format
> specified. Ensure your InputFormat does not require one.
> 14/11/17 09:57:47 INFO utils.ConfigurationUtils: No edge output format
> specified. Ensure your OutputFormat does not require one.
> 14/11/17 09:57:48 INFO yarn.GiraphYarnClient: Final output path is:
> hdfs://localhost:9000/user/hduser/output/shortestpaths
> 14/11/17 09:57:48 INFO yarn.GiraphYarnClient: Running Client
> 14/11/17 09:57:48 INFO client.RMProxy: Connecting to ResourceManager at
> localhost/127.0.0.1:8050
> 14/11/17 09:57:49 INFO yarn.GiraphYarnClient: Got node report from ASM
> for, nodeId=roosevelt:34304, nodeAddress roosevelt:8042, nodeRackName
> /default-rack, nodeNumContainers 0
> 14/11/17 09:57:49 INFO yarn.GiraphYarnClient: Defaulting per-task heap
> size to 1024MB.
> 14/11/17 09:57:49 INFO yarn.GiraphYarnClient: Obtained new Application ID:
> application_1416227900862_0011
> 14/11/17 09:57:49 INFO Configuration.deprecation: mapred.job.id is
> deprecated. Instead, use mapreduce.job.id
> 14/11/17 09:57:49 INFO yarn.GiraphYarnClient: Set the environment for the
> application master
> 14/11/17 09:57:49 INFO yarn.GiraphYarnClient: Environment for AM
> :{CLASSPATH=${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*}
> 14/11/17 09:57:49 INFO yarn.GiraphYarnClient: buildLocalResourceMap ....
> 14/11/17 09:57:50 INFO yarn.YarnUtils: Registered file in LocalResources
> ::
> hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application_1416227900862_0011/giraph-conf.xml
> 14/11/17 09:57:50 INFO yarn.GiraphYarnClient: LIB JARS
> :giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
> 14/11/17 09:57:50 INFO yarn.YarnUtils: Class path name .
> 14/11/17 09:57:50 INFO yarn.YarnUtils: base path checking .
> 14/11/17 09:57:51 INFO yarn.YarnUtils: Class path name /usr/local/hadoop
> 14/11/17 09:57:51 INFO yarn.YarnUtils: base path checking /usr/local/hadoop
> 14/11/17 09:57:51 INFO yarn.YarnUtils: Class path name
> /usr/local/hadoop/etc/hadoop
> 14/11/17 09:57:51 INFO yarn.YarnUtils: base path checking
> /usr/local/hadoop/etc/hadoop
> 14/11/17 09:57:51 INFO yarn.YarnUtils: Class path name
> /usr/local/hadoop/share/hadoop/common/lib/*
> 14/11/17 09:57:51 INFO yarn.YarnUtils: base path checking
> /usr/local/hadoop/share/hadoop/common/lib
> 14/11/17 09:57:51 INFO yarn.YarnUtils: Class path name
> /usr/local/hadoop/share/hadoop/common/*
> 14/11/17 09:57:51 INFO yarn.YarnUtils: base path checking
> /usr/local/hadoop/share/hadoop/common
> 14/11/17 09:57:51 INFO yarn.YarnUtils: Class path name
> /usr/local/hadoop/share/hadoop/hdfs
> 14/11/17 09:57:51 INFO yarn.YarnUtils: base path checking
> /usr/local/hadoop/share/hadoop/hdfs
> 14/11/17 09:57:51 INFO yarn.YarnUtils: Class path name
> /usr/local/hadoop/share/hadoop/hdfs/lib/*
> 14/11/17 09:57:51 INFO yarn.YarnUtils: base path checking
> /usr/local/hadoop/share/hadoop/hdfs/lib
> 14/11/17 09:57:51 INFO yarn.YarnUtils: Class path name
> /usr/local/hadoop/share/hadoop/hdfs/*
> 14/11/17 09:57:51 INFO yarn.YarnUtils: base path checking
> /usr/local/hadoop/share/hadoop/hdfs
> 14/11/17 09:57:51 INFO yarn.YarnUtils: Class path name
> /usr/local/hadoop/share/hadoop/yarn/lib/*
> 14/11/17 09:57:51 INFO yarn.YarnUtils: base path checking
> /usr/local/hadoop/share/hadoop/yarn/lib
> 14/11/17 09:57:51 INFO yarn.YarnUtils: Class path name
> /usr/local/hadoop/share/hadoop/yarn/*
> 14/11/17 09:57:51 INFO yarn.YarnUtils: base path checking
> /usr/local/hadoop/share/hadoop/yarn
> 14/11/17 09:57:51 INFO yarn.YarnUtils: Class path name
> /usr/local/hadoop/share/hadoop/mapreduce/lib/*
> 14/11/17 09:57:51 INFO yarn.YarnUtils: base path checking
> /usr/local/hadoop/share/hadoop/mapreduce/lib
> 14/11/17 09:57:51 INFO yarn.YarnUtils: Class path name
> /usr/local/hadoop/share/hadoop/mapreduce/*
> 14/11/17 09:57:51 INFO yarn.YarnUtils: base path checking
> /usr/local/hadoop/share/hadoop/mapreduce
> 14/11/17 09:57:51 INFO yarn.YarnUtils: Class path name
> /usr/local/hadoop/contrib/capacity-scheduler/*.jar
> 14/11/17 09:57:51 INFO yarn.YarnUtils: base path checking
> /usr/local/hadoop/contrib/capacity-scheduler
> 14/11/17 09:57:51 INFO yarn.GiraphYarnClient: ApplicationSumbissionContext
> for GiraphApplicationMaster launch container is populated.
> 14/11/17 09:57:51 INFO yarn.GiraphYarnClient: Submitting application to ASM
> 14/11/17 09:57:51 INFO impl.YarnClientImpl: Submitted application
> application_1416227900862_0011
> 14/11/17 09:57:51 INFO yarn.GiraphYarnClient: Got new appId after
> submission :application_1416227900862_0011
> 14/11/17 09:57:51 INFO yarn.GiraphYarnClient: GiraphApplicationMaster
> container request was submitted to ResourceManager for job: Giraph:
> org.apache.giraph.examples.SimpleShortestPathsComputation
> 14/11/17 09:57:52 INFO yarn.GiraphYarnClient: Giraph:
> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 0.92
> secs
> 14/11/17 09:57:52 INFO yarn.GiraphYarnClient:
> appattempt_1416227900862_0011_000001, State: ACCEPTED, Containers used: 1
> 14/11/17 09:57:56 ERROR yarn.GiraphYarnClient: Giraph:
> org.apache.giraph.examples.SimpleShortestPathsComputation reports FAILED
> state, diagnostics show: Application application_1416227900862_0011 failed
> 2 times due to AM Container for appattempt_1416227900862_0011_000002 exited
> with  exitCode: 1 due to: Exception from container-launch:
> ExitCodeException exitCode=1:
> ExitCodeException exitCode=1:
>     at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>     at org.apache.hadoop.util.Shell.run(Shell.java:455)
>     at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
>     at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
>     at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
>     at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>     at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>     at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>     at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>     at java.lang.Thread.run(Thread.java:745)
>
>
> Container exited with a non-zero exit code 1
> .Failing this attempt.. Failing the application.
> 14/11/17 09:57:56 INFO yarn.GiraphYarnClient: Cleaning up HDFS distributed
> cache directory for Giraph job.
> 14/11/17 09:57:56 INFO yarn.GiraphYarnClient: Completed Giraph:
> org.apache.giraph.examples.SimpleShortestPathsComputation: FAILED, total
> running time: 0 minutes, 4 seconds.
> Finished hduser script ...
>
> And here is the log for the container:
>
> Error: Could not find or load main class org.apache.giraph.yarn.GiraphApplicationMaster
>
>
>
> On Wed, Nov 19, 2014 at 3:44 PM, Alessandro Negro <alenegro81@yahoo.it>
> wrote:
>
>> Hi Doug,
>> could you paste here the error from hadoop?
>>
>>
>> Alessandro
>>
>> Il giorno 19/nov/2014, alle ore 21:53, Doug Needham <
>> doug.needham@ilwllc.com> ha scritto:
>>
>> I think I may be having a similar problem. Although it could be even more
>> rudimentary than that.
>> I am running CDH5.2 cluster, RHEL 6.4
>> Giraph 1.1.0 (built form github today via: mvn -DskipTests
>> -Dhadoop.version=2.3.0-cdh5.0.0 -Phadoop_yarn clean install)
>>
>>
>> Command I am running is:
>> hadoop jar
>> /usr/local/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.3.0-cdh5.0.0-jar-with-dependencies.jar
>>  org.apache.giraph.GiraphRunner
>> org.apache.giraph.examples.SimplePageRankComputation
>>  -vif
>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>>  -vip /user/dln/ginput/tiny_graph.txt
>>  -vof
>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexOutputFormat
>>  -op /user/dln/goutput2 -w 1
>>
>> It runs for a few minutes then I get the following on stdout:
>>
>> 14/11/19 15:48:16 INFO yarn.GiraphYarnClient:
>> appattempt_1416424267079_0012_000002, State: ACCEPTED, Containers used: 0
>> 14/11/19 15:48:18 ERROR yarn.GiraphYarnClient: Giraph:
>> org.apache.giraph.examples.SimplePageRankComputation reports FAILED state,
>> diagnostics show: Ap
>> plication application_1416424267079_0012 failed 2 times due to AM
>> Container for appattempt_1416424267079_0012_000002 exited with  exitCode: 1
>> due to:
>> Exception from container-launch.
>> Container id: container_1416424267079_0012_02_000001
>> Exit code: 1
>> Stack trace: ExitCodeException exitCode=1:
>>         at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>>         at org.apache.hadoop.util.Shell.run(Shell.java:455)
>>         at
>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
>>         at
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196)
>>         at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
>>         at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>         at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>         at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>         at java.lang.Thread.run(Thread.java:745)
>>
>>
>> Container exited with a non-zero exit code 1
>> .Failing this attempt.. Failing the application.
>> 14/11/19 15:48:18 INFO yarn.GiraphYarnClient: Cleaning up HDFS
>> distributed cache directory for Giraph job.
>> 14/11/19 15:48:18 INFO yarn.GiraphYarnClient: Completed Giraph:
>> org.apache.giraph.examples.SimplePageRankComputation: FAILED, total running
>> time: 0 mi
>> nutes, 14 seconds.
>>
>>
>>
>> Any suggestions?
>>
>> Thank you,
>>
>> Doug
>>
>>
>>
>>
>> *From:* adamosdas@gmail.com [mailto:adamosdas@gmail.com
>> <adamosdas@gmail.com>] *On Behalf Of *D Adams
>> *Sent:* Wednesday, November 19, 2014 3:34 PM
>> *To:* user@giraph.apache.org
>> *Subject:* Re: Help with Giraph on Yarn
>>
>> Alessandro,
>>
>>      Sorry I didn't get back to you. I have 3 VMs right now and have
>> gotten confused about which hadoop/giraph configuration I have on each. I
>> will get back to this thread once I have regained awareness of which VM has
>> that error. Thanks,
>> V/r
>> Das
>>
>> On Wed, Nov 19, 2014 at 9:54 AM, Tripti Singh <tripti@yahoo-inc.com>
>> wrote:
>>
>> Hi Alessandro,
>>
>> I guess u r talking about the documentation on Giraph webpage which
>> mentions addition of this new option which is more or less mandatory for
>> yarn based profiles.
>> When I first ran the Giraph on yarn without the proper –yj option, there
>> was no way I could figure out from the Application/Container logs that this
>> was the issue.
>> I think, it’ll be a good idea to have this message in the logs for easy
>> debugging.
>>
>> Thanks,
>> Tripti.
>>
>>
>> *From: *Alessandro Negro <alenegro81@yahoo.it>
>> *Reply-To: *"user@giraph.apache.org" <user@giraph.apache.org>
>> *Date: *Tuesday, November 18, 2014 at 4:31 PM
>>
>> *To: *"user@giraph.apache.org" <user@giraph.apache.org>
>> *Subject: *Re: Help with Giraph on Yarn
>>
>> Hi Eli,
>> no I think that the help message is clear enough, generally when I read
>> “jar” I always mean absolute path for jar, but this is a my personal
>> misunderstanding.
>>
>> Thanks,
>> Alessandro
>>
>>
>> Il giorno 16/nov/2014, alle ore 21:39, Eli Reisman <
>> apache.mailbox@gmail.com> ha scritto:
>>
>>
>> Sounds like you got it figured out since last time I checked this list,
>> sorry it was a pain. Feel free to drop a JIRA ticket if we can make help
>> message etc. for -yj option more clear, there's lots to improve there.
>>
>> On Sat, Nov 8, 2014 at 7:26 AM, Alessandro Negro <alenegro81@yahoo.it>
>> wrote:
>>
>> Hi Tripti,
>> finally I was able to run the test with success. It was an issue of
>> permission since I was running as ale not as yarn.
>> Let me say that now I’m able to run graph examples on Yarn 2.5.1. This is
>> the final result:
>>
>> 14/11/08 16:24:00 INFO yarn.GiraphYarnClient: Completed Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation: SUCCEEDED, total
>> running time: 0 minutes, 21 seconds.
>>
>>
>> Many thanks for your support,
>> Alessandro
>>
>> Il giorno 06/nov/2014, alle ore 15:16, Tripti Singh <tripti@yahoo-inc.com>
>> ha scritto:
>>
>>
>> I don't know if u have access to this node. But if u do, u could check if
>> the file is indeed there and u have access to it.
>>
>> Sent from my iPhone
>>
>>
>> On 06-Nov-2014, at 6:12 pm, "Alessandro Negro" <alenegro81@yahoo.it>
>> wrote:
>>
>> You are right it works, but now I receive the following error:
>>
>> SLF4J: Class path contains multiple SLF4J bindings.
>> SLF4J: Found binding in
>> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/usercache/ale/appcache/application_1415264041937_0009/filecache/10/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: Found binding in
>> [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>> explanation.
>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>> 2014-11-06 13:15:37.120 java[10158:1803] Unable to load realm info from
>> SCDynamicStore
>> Exception in thread "pool-4-thread-1" java.lang.IllegalStateException:
>> Could not configure the containerlaunch context for GiraphYarnTasks.
>> at
>> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:391)
>> at
>> org.apache.giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicationMaster.java:78)
>> at
>> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.buildContainerLaunchContext(GiraphApplicationMaster.java:522)
>> at
>> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.run(GiraphApplicationMaster.java:479)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:744)
>> Caused by: java.io.FileNotFoundException: File does not exist:
>> hdfs://hadoop-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_0009/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>> at
>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1072)
>> at
>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1064)
>> at
>> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>> at
>> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1064)
>> at
>> org.apache.giraph.yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153)
>> at org.apache.giraph.yarn.YarnUtils.addFsResourcesToMap(YarnUtils.java:77)
>> at
>> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:387)
>> ... 6 more
>>
>>
>> That justify the other error I receive in the task:
>>
>> Could not find or load main class org.apache.giraph.yarn.GiraphYarnTask
>>
>>
>> Thanks,
>>
>> Il giorno 06/nov/2014, alle ore 13:07, Tripti Singh <tripti@yahoo-inc.com>
>> ha scritto:
>>
>>
>> Why r u adding two jars? Example jar ideally contains core library so
>> everything should be available with just one example jar included
>>
>> Sent from my iPhone
>>
>>
>> On 06-Nov-2014, at 4:33 pm, "Alessandro Negro" <alenegro81@yahoo.it>
>> wrote:
>>
>> Hi,
>> now it seems better, I need to add:
>>
>>
>> giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>>
>>
>> Now it seems that after a lot of cycle it fail with this error:
>>
>> Could not find or load main class org.apache.giraph.yarn.GiraphYarnTask
>>
>> But in this case the error appear in task-3-stderr.log  not
>> in gam-stderr.log where there is the following error:
>>
>> LF4J: Class path contains multiple SLF4J bindings.
>> SLF4J: Found binding in
>> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/usercache/ale/appcache/application_1415264041937_0006/filecache/12/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: Found binding in
>> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/usercache/ale/appcache/application_1415264041937_0006/filecache/10/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: Found binding in
>> [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>> explanation.
>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>> 2014-11-06 11:31:25.186 java[8162:1803] Unable to load realm info from
>> SCDynamicStore
>> Exception in thread "pool-4-thread-1" java.lang.IllegalStateException:
>> Could not configure the containerlaunch context for GiraphYarnTasks.
>> at
>> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:391)
>> at
>> org.apache.giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicationMaster.java:78)
>> at
>> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.buildContainerLaunchContext(GiraphApplicationMaster.java:522)
>> at
>> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.run(GiraphApplicationMaster.java:479)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:744)
>> Caused by: java.io.FileNotFoundException: File does not exist:
>> hdfs://hadoop-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_0006/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>> at
>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1072)
>> at
>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1064)
>> at
>> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>> at
>> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1064)
>> at
>> org.apache.giraph.yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153)
>> at org.apache.giraph.yarn.YarnUtils.addFsResourcesToMap(YarnUtils.java:77)
>> at
>> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:387)
>> ... 6 more
>>
>>
>> Il giorno 06/nov/2014, alle ore 11:32, Tripti Singh <tripti@yahoo-inc.com>
>> ha scritto:
>>
>>
>> Hi,
>> I remember seeing this thread, but since i was already deep into fixing
>> this issue in production, I didn't really wait for the patch, not even the
>> munge symbol as I was not even able to compile the yarn profile( sorry for
>> being so ruthless to giraph :-( )
>> I did later cross check with the patch and it was quite similar to my
>> change but sure it wouldn't work for all profiles(in fact some profiles may
>> even break due to Hadoop class incompatibility ) I can share those changes
>> with u once I get back to my computer but I don't think that could have
>> caused this problem.
>> Just try the small changed I mentioned in my previous reply and let me
>> know if that helped.
>>
>> Thanks,
>> Tripti
>> Sent from my iPhone
>>
>>
>> On 06-Nov-2014, at 3:52 pm, "Alessandro Negro" <alenegro81@yahoo.it>
>> wrote:
>>
>> Hi Tripti,
>> let me add one info, As you can see from this thread of email:
>>
>>
>> http://mail-archives.apache.org/mod_mbox/giraph-user/201410.mbox/%3CC3F75A98-BDE8-4205-971F-1F7241E6E83F@yahoo.it%3E
>>
>> I had some issue while compiling for Hadoop 2.5.1 could be that into the
>> process I loose something?
>>
>> Thanks again,
>> Alessandro
>>
>>
>>
>> Il giorno 06/nov/2014, alle ore 10:47, Alessandro Negro <
>> alenegro81@yahoo.it> ha scritto:
>>
>>
>> Hi Tripti,
>> yes I run in this way:
>>
>>
>> $HADOOP_INSTALL/bin/hadoop jar
>> /Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner
>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>> -vip /tiny_graph.txt -vof
>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /shortestpaths
>> -w 1 -yj
>> /Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>>
>> And this is the complete trace:
>>
>> 2014-11-06 10:06:05.638 java[7778:1903] Unable to load realm info from
>> SCDynamicStore
>> 14/11/06 10:06:05 WARN util.NativeCodeLoader: Unable to load
>> native-hadoop library for your platform... using builtin-java classes where
>> applicable
>> 14/11/06 10:06:06 INFO utils.ConfigurationUtils: No edge input format
>> specified. Ensure your InputFormat does not require one.
>> 14/11/06 10:06:06 INFO utils.ConfigurationUtils: No edge output format
>> specified. Ensure your OutputFormat does not require one.
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Final output path is:
>> hdfs://hadoop-master:9000/shortestpaths
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Running Client
>> 14/11/06 10:06:06 INFO client.RMProxy: Connecting to ResourceManager at /
>> 0.0.0.0:8032
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got node report from ASM
>> for, nodeId=10.61.137.28:59677, nodeAddress 10.61.137.28:8042,
>> nodeRackName /default-rack, nodeNumContainers 0
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Defaulting per-task heap
>> size to 1024MB.
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Obtained new Application
>> ID: application_1415264041937_0005
>> 14/11/06 10:06:06 INFO Configuration.deprecation: mapred.job.id is
>> deprecated. Instead, use mapreduce.job.id
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Set the environment for the
>> application master
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Environment for AM
>> :{CLASSPATH=${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*}
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: buildLocalResourceMap ....
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Registered file in LocalResources
>> ::
>> hdfs://hadoop-master:9000/user/ale/giraph_yarn_jar_cache/application_1415264041937_0005/giraph-conf.xml
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: LIB JARS
>> :/Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name .
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking .
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name null
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking null
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/etc/hadoop
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/etc/hadoop
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/*
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/*
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/common
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/*
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/*
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib/*
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib/*
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/*
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/myLib/*.jar
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/myLib
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /Users/ale/extprj/giraph/myGiraph/*.jar
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /Users/ale/extprj/giraph/myGiraph
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /Users/ale/extprj/giraph/myGiraph/lib/*.jar
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /Users/ale/extprj/giraph/myGiraph/lib
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /contrib/capacity-scheduler/*.jar
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /contrib/capacity-scheduler
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient:
>> ApplicationSumbissionContext for GiraphApplicationMaster launch container
>> is populated.
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application to
>> ASM
>> 14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application
>> application_1415264041937_0005
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after
>> submission :application_1415264041937_0005
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: GiraphApplicationMaster
>> container request was submitted to ResourceManager for job: Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation
>> 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 0,83
>> secs
>> 14/11/06 10:06:07 INFO yarn.GiraphYarnClient:
>> appattempt_1415264041937_0005_000001, State: ACCEPTED, Containers used: 1
>> 14/11/06 10:06:10 ERROR yarn.GiraphYarnClient: Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation reports FAILED
>> state, diagnostics show: Application application_1415264041937_0005 failed
>> 2 times due to AM Container for appattempt_1415264041937_0005_000002 exited
>> with  exitCode: 1 due to: Exception from container-launch:
>> ExitCodeException exitCode=1:
>> ExitCodeException exitCode=1:
>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>> at org.apache.hadoop.util.Shell.run(Shell.java:455)
>> at
>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:744)
>>
>>
>> Container exited with a non-zero exit code 1
>> .Failing this attempt.. Failing the application.
>> 14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Cleaning up HDFS
>> distributed cache directory for Giraph job.
>> 14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Completed Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation: FAILED, total
>> running time: 0 minutes, 3 seconds.
>> ale:~/extprj/giraph/myGiraph />yarn logs -applicationId
>> application_1415264041937_0005
>>
>> As you can see there is this piece:
>>
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: LIB JARS
>> :/Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>>
>>
>> But I continue to receive the error:
>>
>> Thanks,
>> Alessandro
>>
>>
>> Il giorno 06/nov/2014, alle ore 10:25, Tripti Singh <tripti@yahoo-inc.com>
>> ha scritto:
>>
>>
>> For GiraphApplicationMaster not found, the simple thing that worked for
>> me was adding a -yj parameter to my command.
>> (hadoop jar {jar-file-path} {path to giraph runner} {computation-class}
>> {bunch of parameters} -yj {jar-file-name})
>> If u look at job logs,
>> Do u see a log statement defining "LIB JARS"?
>>
>> Sent from my iPhone
>>
>>
>> On 06-Nov-2014, at 2:45 pm, "Alessandro Negro" <alenegro81@yahoo.it>
>> wrote:
>>
>> Hi,
>> yea my logs shows that error. I follow this example:
>>
>>
>> http://mail-archives.apache.org/mod_mbox/giraph-user/201408.mbox/%3C53F4C689.5060101@web.de%3E
>>
>> without success.
>> Then I try to add all jars I can from giraph adding the following lines
>> into the hadoop-env.sh:
>>
>> export
>> MYLIB=/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar
>> export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$MYLIB
>>
>>
>> when I run the application now I see this:
>>
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/myLib/*.jar
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/myLib
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /Users/ale/extprj/giraph/myGiraph/*.jar
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /Users/ale/extprj/giraph/myGiraph
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /Users/ale/extprj/giraph/myGiraph/lib/*.jar
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /Users/ale/extprj/giraph/myGiraph/lib
>>
>>
>> But again the same error. What I think is that there is no problem in the
>> library but while is starting up.
>> I saw this piece in the logs:
>>
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient:
>> ApplicationSumbissionContext for GiraphApplicationMaster launch container
>> is populated.
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application to
>> ASM
>> 14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application
>> application_1415264041937_0005
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after
>> submission :application_1415264041937_0005
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: GiraphApplicationMaster
>> container request was submitted to ResourceManager for job: Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation
>> 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 0,83
>> secs
>>
>>
>> That show that it is able to load GiraphApplicationMaster but fail during
>> the startup.
>> Where can I find more logs?
>>
>> Thanks,
>> Alessandro
>>
>>
>> Il giorno 06/nov/2014, alle ore 06:56, Tripti Singh <tripti@yahoo-inc.com>
>> ha scritto:
>>
>>
>> Can u run the following command to find container logs and post it here:
>>
>> yarn logs -applicationId application_1415203231170_0002
>>
>> If u find this error:
>>
>> Could not find or load main class org.apache.giraph.yarn.GiraphApplicationMaster
>>
>> It may just be due to non-availablity of the yarn jars
>>
>> Thanks,
>> *Tripti*
>> Tech Yahoo
>>
>> *From: *Jing Fan <fanjing09@gmail.com>
>> *Reply-To: *"user@giraph.apache.org" <user@giraph.apache.org>
>> *Date: *Thursday, November 6, 2014 at 1:56 AM
>> *To: *"user@giraph.apache.org" <user@giraph.apache.org>
>> *Subject: *Re: Help with Giraph on Yarn
>>
>> Then do you know to solve this error?
>>
>> 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation reports FAILED
>> state, diagnostics show: Application application_1415203231170_0002 failed
>> 2 times due to AM Container for appattempt_1415203231170_0002_000002 exited
>> with  exitCode: 1 due to: Exception from container-launch.
>> Container id: container_1415203231170_0002_02_000001
>> Exit code: 1
>> Stack trace: ExitCodeException exitCode=1:
>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>> at org.apache.hadoop.util.Shell.run(Shell.java:455)
>> at
>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:745)
>>
>>
>> On Wed, Nov 5, 2014 at 12:10 PM, Alessandro Negro <alenegro81@yahoo.it>
>> wrote:
>>
>> Hi,
>> in the mean time I found the reason why it hangs for long time here:
>>
>> base path checking .
>>
>>
>> It scan all the directory tree starting from current directory, if you
>> run it into your home (and you have a lot of files and directories) it
>> takes ages to complete.
>>
>> I continue to got the same error.
>>
>> Regards,
>> Alessandro
>>
>> Il giorno 05/nov/2014, alle ore 18:33, Jing Fan <fanjing09@gmail.com> ha
>> scritto:
>>
>>
>> I run with:
>>
>> hadoop jar
>> giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner
>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>> -vip /user/jing/tiny_graph.txt -vof
>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>> /user/jing/shortestpaths -w 1 -yj
>> giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar
>>
>> On Wed, Nov 5, 2014 at 11:25 AM, Alessandro Negro <alenegro81@yahoo.it>
>> wrote:
>>
>> This is my complete logs with the command I run:
>>
>>
>> ale:~ />$HADOOP_INSTALL/bin/hadoop jar
>> /Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner
>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>> -vip /tiny_graph.txt -vof
>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /shortestpaths
>> -w 1
>> 2014-11-05 13:39:30.235 java[3147:1903] Unable to load realm info from
>> SCDynamicStore
>> 14/11/05 13:39:30 WARN util.NativeCodeLoader: Unable to load
>> native-hadoop library for your platform... using builtin-java classes where
>> applicable
>> 14/11/05 13:39:30 INFO utils.ConfigurationUtils: No edge input format
>> specified. Ensure your InputFormat does not require one.
>> 14/11/05 13:39:30 INFO utils.ConfigurationUtils: No edge output format
>> specified. Ensure your OutputFormat does not require one.
>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Final output path is:
>> hdfs://hadoop-master:9000/shortestpaths
>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Running Client
>> 14/11/05 13:39:31 INFO client.RMProxy: Connecting to ResourceManager at /
>> 0.0.0.0:8032
>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Got node report from ASM
>> for, nodeId=192.168.0.105:50658, nodeAddress192.168.0.105:8042,
>> nodeRackName /default-rack, nodeNumContainers 0
>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Defaulting per-task heap
>> size to 1024MB.
>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Obtained new Application
>> ID: application_1415189069846_0004
>> 14/11/05 13:39:31 INFO Configuration.deprecation: mapred.job.id is
>> deprecated. Instead, use mapreduce.job.id
>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Set the environment for the
>> application master
>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Environment for AM
>> :{CLASSPATH=${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*}
>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: buildLocalResourceMap ....
>> 14/11/05 13:39:31 INFO yarn.YarnUtils: Registered file in LocalResources
>> ::
>> hdfs://hadoop-master:9000/user/ale/giraph_yarn_jar_cache/application_1415189069846_0004/giraph-conf.xml
>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: LIB JARS :
>> 14/11/05 13:39:31 INFO yarn.YarnUtils: Class path name .
>> 14/11/05 13:39:31 INFO yarn.YarnUtils: base path checking .
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name null
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking null
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/etc/hadoop
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/etc/hadoop
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/*
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/*
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/common
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/*
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/*
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib/*
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib/*
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/*
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>> /contrib/capacity-scheduler/*.jar
>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>> /contrib/capacity-scheduler
>> 14/11/05 14:42:56 INFO yarn.GiraphYarnClient:
>> ApplicationSumbissionContext for GiraphApplicationMaster launch container
>> is populated.
>> 14/11/05 14:42:56 INFO yarn.GiraphYarnClient: Submitting application to
>> ASM
>> 14/11/05 14:42:57 INFO impl.YarnClientImpl: Submitted application
>> application_1415189069846_0004
>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: Got new appId after
>> submission :application_1415189069846_0004
>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: GiraphApplicationMaster
>> container request was submitted to ResourceManager for job: Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation
>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 0,82
>> secs
>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient:
>> appattempt_1415189069846_0004_000001, State: ACCEPTED, Containers used: 1
>> 14/11/05 14:43:01 ERROR yarn.GiraphYarnClient: Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation reports FAILED
>> state, diagnostics show: Application application_1415189069846_0004 failed
>> 2 times due to AM Container for appattempt_1415189069846_0004_000002 exited
>> with  exitCode: 1 due to: Exception from container-launch:
>> ExitCodeException exitCode=1:
>> ExitCodeException exitCode=1:
>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>> at org.apache.hadoop.util.Shell.run(Shell.java:455)
>> at
>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:744)
>>
>>
>> Container exited with a non-zero exit code 1
>> .Failing this attempt.. Failing the application.
>> 14/11/05 14:43:01 INFO yarn.GiraphYarnClient: Cleaning up HDFS
>> distributed cache directory for Giraph job.
>> 14/11/05 14:43:01 INFO yarn.GiraphYarnClient: Completed Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation: FAILED, total
>> running time: 0 minutes, 3 seconds.
>>
>>
>> Il giorno 05/nov/2014, alle ore 18:17, Tripti Singh <tripti@yahoo-inc.com>
>> ha scritto:
>>
>>
>> I faced the similar problem where the workflow just hangs.
>> Can u see in ur logs if there is a task with id 0.
>> I made a local fix and I could give u the patch if u see a similar
>> problem and ur workflow is unable to find the zookeeper manager file and
>> repeats the following message:
>> For task xyz, got file 'null'
>>
>> Thanks,
>> Tripti
>> Sent from my iPhone
>>
>>
>> On 05-Nov-2014, at 10:40 pm, "Alessandro Negro" <alenegro81@yahoo.it>
>> wrote:
>>
>> Hi,
>> have you seen the hadoop logs?
>> Could you write here how are you running the command?
>>
>> Thanks,
>> Alessandro
>>
>> Il giorno 05/nov/2014, alle ore 18:04, Jing Fan <fanjing09@gmail.com> ha
>> scritto:
>>
>>
>> Yeah, I add -yj option. The result is the same. Still error here.
>>
>> On Wed, Nov 5, 2014 at 10:52 AM, Alessandro Negro <alenegro81@yahoo.it>
>> wrote:
>>
>> Hi Jing,
>> I’m facing the same problem. Do you check the hadoop logs? In my case I
>> got:
>>
>> Could not find or load main class org.apache.giraph.yarn.GiraphApplicationMaster
>>
>> Moreover I don’t know why my application hang for a long time here:
>>
>> 14/11/05 17:11:17 INFO yarn.YarnUtils: base path checking .
>>
>> And then continue failing with your error into the output and the other
>> into the hadoop logs.
>>
>> Did you test adding this?
>>
>> -yj
>> /Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>>
>> Thanks,
>> Alessandro
>>
>>
>> Il giorno 05/nov/2014, alle ore 17:32, Jing Fan <fanjing09@gmail.com> ha
>> scritto:
>>
>>
>> Hi,
>>
>> I install the Giraph with Yarn.
>> But when I run the program with following command:
>>
>>  hadoop jar
>> giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner
>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>> -vip /user/j/tiny_graph.txt -vof
>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>> /user/j/shortestpaths -w 1
>>
>> The program exited with error:
>>
>> 14/11/05 10:25:14 INFO yarn.GiraphYarnClient: Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 0.83
>> secs
>> 14/11/05 10:25:14 INFO yarn.GiraphYarnClient:
>> appattempt_1415203231170_0002_000001, State: ACCEPTED, Containers used: 0
>> 14/11/05 10:25:18 INFO yarn.GiraphYarnClient: Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 4.84
>> secs
>> 14/11/05 10:25:18 INFO yarn.GiraphYarnClient:
>> appattempt_1415203231170_0002_000001, State: ACCEPTED, Containers used: 0
>> 14/11/05 10:25:22 INFO yarn.GiraphYarnClient: Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 8.85
>> secs
>> 14/11/05 10:25:22 INFO yarn.GiraphYarnClient:
>> appattempt_1415203231170_0002_000002, State: ACCEPTED, Containers used: 0
>> 14/11/05 10:25:26 INFO yarn.GiraphYarnClient: Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 12.86
>> secs
>> 14/11/05 10:25:26 INFO yarn.GiraphYarnClient:
>> appattempt_1415203231170_0002_000002, State: ACCEPTED, Containers used: 0
>> 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation reports FAILED
>> state, diagnostics show: Application application_1415203231170_0002 failed
>> 2 times due to AM Container for appattempt_1415203231170_0002_000002 exited
>> with  exitCode: 1 due to: Exception from container-launch.
>> Container id: container_1415203231170_0002_02_000001
>> Exit code: 1
>> Stack trace: ExitCodeException exitCode=1:
>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>> at org.apache.hadoop.util.Shell.run(Shell.java:455)
>> at
>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:745)
>>
>>
>> Anyone can help?
>>
>>
>>
>
>

Mime
View raw message