giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alessandro Negro <alenegr...@yahoo.it>
Subject Re: Help with Giraph on Yarn
Date Thu, 06 Nov 2014 09:14:09 GMT
Hi,
yea my logs shows that error. I follow this example:

http://mail-archives.apache.org/mod_mbox/giraph-user/201408.mbox/%3C53F4C689.5060101@web.de%3E

without success.
Then I try to add all jars I can from giraph adding the following lines into the hadoop-env.sh:

export MYLIB=/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$MYLIB


when I run the application now I see this:

14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name 
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5.1/share/myLib/*.jar
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-2.5.1/share/myLib
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /Users/ale/extprj/giraph/myGiraph/*.jar
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /Users/ale/extprj/giraph/myGiraph
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /Users/ale/extprj/giraph/myGiraph/lib/*.jar
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /Users/ale/extprj/giraph/myGiraph/lib


But again the same error. What I think is that there is no problem in the library but while
is starting up.
I saw this piece in the logs:

14/11/06 10:06:06 INFO yarn.GiraphYarnClient: ApplicationSumbissionContext for GiraphApplicationMaster
launch container is populated.
14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application to ASM
14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application application_1415264041937_0005
14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after submission :application_1415264041937_0005
14/11/06 10:06:06 INFO yarn.GiraphYarnClient: GiraphApplicationMaster container request was
submitted to ResourceManager for job: Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation
14/11/06 10:06:07 INFO yarn.GiraphYarnClient: Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation,
Elapsed: 0,83 secs


That show that it is able to load GiraphApplicationMaster but fail during the startup.
Where can I find more logs?

Thanks,
Alessandro


Il giorno 06/nov/2014, alle ore 06:56, Tripti Singh <tripti@yahoo-inc.com> ha scritto:

> Can u run the following command to find container logs and post it here:
> 
> yarn logs -applicationId application_1415203231170_0002
> 
> If u find this error:
> Could not find or load main class org.apache.giraph.yarn.GiraphApplicationMaster
> It may just be due to non-availablity of the yarn jars
> Thanks,
> Tripti
> Tech Yahoo
> 
> From: Jing Fan <fanjing09@gmail.com>
> Reply-To: "user@giraph.apache.org" <user@giraph.apache.org>
> Date: Thursday, November 6, 2014 at 1:56 AM
> To: "user@giraph.apache.org" <user@giraph.apache.org>
> Subject: Re: Help with Giraph on Yarn
> 
> Then do you know to solve this error?
> 
> 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation
reports FAILED state, diagnostics show: Application application_1415203231170_0002 failed
2 times due to AM Container for appattempt_1415203231170_0002_000002 exited with  exitCode:
1 due to: Exception from container-launch.
> Container id: container_1415203231170_0002_02_000001
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1: 
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
> at org.apache.hadoop.util.Shell.run(Shell.java:455)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 
> 
> On Wed, Nov 5, 2014 at 12:10 PM, Alessandro Negro <alenegro81@yahoo.it> wrote:
>> Hi,
>> in the mean time I found the reason why it hangs for long time here:
>> 
>> base path checking .
>> 
>> 
>> It scan all the directory tree starting from current directory, if you run it into
your home (and you have a lot of files and directories) it takes ages to complete.
>> 
>> I continue to got the same error.
>> 
>> Regards,
>> Alessandro
>> 
>> Il giorno 05/nov/2014, alle ore 18:33, Jing Fan <fanjing09@gmail.com> ha scritto:
>> 
>>> I run with:
>>> 
>>> hadoop jar giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/jing/tiny_graph.txt
-vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/jing/shortestpaths
-w 1 -yj giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar
>>> 
>>> On Wed, Nov 5, 2014 at 11:25 AM, Alessandro Negro <alenegro81@yahoo.it>
wrote:
>>>> This is my complete logs with the command I run:
>>>> 
>>>> 
>>>> ale:~ />$HADOOP_INSTALL/bin/hadoop jar /Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /tiny_graph.txt
-vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /shortestpaths -w 1
>>>> 2014-11-05 13:39:30.235 java[3147:1903] Unable to load realm info from SCDynamicStore
>>>> 14/11/05 13:39:30 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
>>>> 14/11/05 13:39:30 INFO utils.ConfigurationUtils: No edge input format specified.
Ensure your InputFormat does not require one.
>>>> 14/11/05 13:39:30 INFO utils.ConfigurationUtils: No edge output format specified.
Ensure your OutputFormat does not require one.
>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Final output path is: hdfs://hadoop-master:9000/shortestpaths
>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Running Client
>>>> 14/11/05 13:39:31 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Got node report from ASM for,
nodeId=192.168.0.105:50658, nodeAddress 192.168.0.105:8042, nodeRackName /default-rack, nodeNumContainers
0
>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Defaulting per-task heap size
to 1024MB.
>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Obtained new Application ID:
application_1415189069846_0004
>>>> 14/11/05 13:39:31 INFO Configuration.deprecation: mapred.job.id is deprecated.
Instead, use mapreduce.job.id
>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Set the environment for the
application master
>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Environment for AM :{CLASSPATH=${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*}
>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: buildLocalResourceMap ....
>>>> 14/11/05 13:39:31 INFO yarn.YarnUtils: Registered file in LocalResources
:: hdfs://hadoop-master:9000/user/ale/giraph_yarn_jar_cache/application_1415189069846_0004/giraph-conf.xml
>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: LIB JARS :
>>>> 14/11/05 13:39:31 INFO yarn.YarnUtils: Class path name .
>>>> 14/11/05 13:39:31 INFO yarn.YarnUtils: base path checking .
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name null
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking null
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5.1/etc/hadoop
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-2.5.1/etc/hadoop
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/*
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5.1/share/hadoop/common/*
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-2.5.1/share/hadoop/common
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/*
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/*
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib/*
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-2.5.1/share/hadoop/yarn
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib/*
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/*
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name /contrib/capacity-scheduler/*.jar
>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking /contrib/capacity-scheduler
>>>> 14/11/05 14:42:56 INFO yarn.GiraphYarnClient: ApplicationSumbissionContext
for GiraphApplicationMaster launch container is populated.
>>>> 14/11/05 14:42:56 INFO yarn.GiraphYarnClient: Submitting application to ASM
>>>> 14/11/05 14:42:57 INFO impl.YarnClientImpl: Submitted application application_1415189069846_0004
>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: Got new appId after submission
:application_1415189069846_0004
>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: GiraphApplicationMaster container
request was submitted to ResourceManager for job: Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation
>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation,
Elapsed: 0,82 secs
>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: appattempt_1415189069846_0004_000001,
State: ACCEPTED, Containers used: 1
>>>> 14/11/05 14:43:01 ERROR yarn.GiraphYarnClient: Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation
reports FAILED state, diagnostics show: Application application_1415189069846_0004 failed
2 times due to AM Container for appattempt_1415189069846_0004_000002 exited with  exitCode:
1 due to: Exception from container-launch: ExitCodeException exitCode=1: 
>>>> ExitCodeException exitCode=1: 
>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455)
>>>> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
>>>> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
>>>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
>>>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>> at java.lang.Thread.run(Thread.java:744)
>>>> 
>>>> 
>>>> Container exited with a non-zero exit code 1
>>>> .Failing this attempt.. Failing the application.
>>>> 14/11/05 14:43:01 INFO yarn.GiraphYarnClient: Cleaning up HDFS distributed
cache directory for Giraph job.
>>>> 14/11/05 14:43:01 INFO yarn.GiraphYarnClient: Completed Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation:
FAILED, total running time: 0 minutes, 3 seconds.
>>>> 
>>>> 
>>>> Il giorno 05/nov/2014, alle ore 18:17, Tripti Singh <tripti@yahoo-inc.com>
ha scritto:
>>>> 
>>>>> I faced the similar problem where the workflow just hangs. 
>>>>> Can u see in ur logs if there is a task with id 0.
>>>>> I made a local fix and I could give u the patch if u see a similar problem
and ur workflow is unable to find the zookeeper manager file and repeats the following message:
>>>>> For task xyz, got file 'null'
>>>>> 
>>>>> Thanks,
>>>>> Tripti
>>>>> Sent from my iPhone
>>>>> 
>>>>> On 05-Nov-2014, at 10:40 pm, "Alessandro Negro" <alenegro81@yahoo.it>
wrote:
>>>>> 
>>>>>> Hi,
>>>>>> have you seen the hadoop logs?
>>>>>> Could you write here how are you running the command?
>>>>>> 
>>>>>> Thanks,
>>>>>> Alessandro
>>>>>> 
>>>>>> Il giorno 05/nov/2014, alle ore 18:04, Jing Fan <fanjing09@gmail.com>
ha scritto:
>>>>>> 
>>>>>>> Yeah, I add -yj option. The result is the same. Still error here.
>>>>>>> 
>>>>>>> On Wed, Nov 5, 2014 at 10:52 AM, Alessandro Negro <alenegro81@yahoo.it>
wrote:
>>>>>>>> Hi Jing,
>>>>>>>> I’m facing the same problem. Do you check the hadoop logs?
In my case I got: 
>>>>>>>> Could not find or load main class org.apache.giraph.yarn.GiraphApplicationMaster
>>>>>>>> Moreover I don’t know why my application hang for a long
time here:
>>>>>>>> 
>>>>>>>> 14/11/05 17:11:17 INFO yarn.YarnUtils: base path checking
.
>>>>>>>> 
>>>>>>>> And then continue failing with your error into the output
and the other into the hadoop logs.
>>>>>>>> 
>>>>>>>> Did you test adding this?
>>>>>>>> 
>>>>>>>> -yj /Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Alessandro 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Il giorno 05/nov/2014, alle ore 17:32, Jing Fan <fanjing09@gmail.com>
ha scritto:
>>>>>>>> 
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> I install the Giraph with Yarn.
>>>>>>>>> But when I run the program with following command:
>>>>>>>>> 
>>>>>>>>>  hadoop jar giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/j/tiny_graph.txt
-vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/j/shortestpaths -w
1
>>>>>>>>> 
>>>>>>>>> The program exited with error:
>>>>>>>>> 
>>>>>>>>> 14/11/05 10:25:14 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 0.83 secs
>>>>>>>>> 14/11/05 10:25:14 INFO yarn.GiraphYarnClient: appattempt_1415203231170_0002_000001,
State: ACCEPTED, Containers used: 0
>>>>>>>>> 14/11/05 10:25:18 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 4.84 secs
>>>>>>>>> 14/11/05 10:25:18 INFO yarn.GiraphYarnClient: appattempt_1415203231170_0002_000001,
State: ACCEPTED, Containers used: 0
>>>>>>>>> 14/11/05 10:25:22 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 8.85 secs
>>>>>>>>> 14/11/05 10:25:22 INFO yarn.GiraphYarnClient: appattempt_1415203231170_0002_000002,
State: ACCEPTED, Containers used: 0
>>>>>>>>> 14/11/05 10:25:26 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 12.86 secs
>>>>>>>>> 14/11/05 10:25:26 INFO yarn.GiraphYarnClient: appattempt_1415203231170_0002_000002,
State: ACCEPTED, Containers used: 0
>>>>>>>>> 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation reports FAILED state, diagnostics
show: Application application_1415203231170_0002 failed 2 times due to AM Container for appattempt_1415203231170_0002_000002
exited with  exitCode: 1 due to: Exception from container-launch.
>>>>>>>>> Container id: container_1415203231170_0002_02_000001
>>>>>>>>> Exit code: 1
>>>>>>>>> Stack trace: ExitCodeException exitCode=1: 
>>>>>>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>>>>>>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455)
>>>>>>>>> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
>>>>>>>>> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196)
>>>>>>>>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
>>>>>>>>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>>>>>>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Anyone can help?
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> 
>>> 
>> 
> 


Mime
View raw message