Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 35B2F10685 for ; Sat, 22 Nov 2014 17:44:50 +0000 (UTC) Received: (qmail 15939 invoked by uid 500); 22 Nov 2014 17:44:50 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 15892 invoked by uid 500); 22 Nov 2014 17:44:49 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 15882 invoked by uid 99); 22 Nov 2014 17:44:49 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 22 Nov 2014 17:44:49 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of dadamszx@gmail.com designates 74.125.82.65 as permitted sender) Received: from [74.125.82.65] (HELO mail-wg0-f65.google.com) (74.125.82.65) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 22 Nov 2014 17:44:45 +0000 Received: by mail-wg0-f65.google.com with SMTP id k14so2103925wgh.0 for ; Sat, 22 Nov 2014 09:42:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=ykITyC3CenQcQPG2LI1eR7KFBA5/ST5CSR9zPDgGaZE=; b=x5wQasgDs092RC+Xg9Vi6HBXCBNT6ojafA2jsX9AN01SycBHEtLpQQ7raVl+p0Zfhl nZ3voPLFTDNQXkNE7w6717IFQZum3iCylfAOH5/TTgzmSYk+LYC114vmsyPCunMY712Y Cwys8IVmWTjXuc4HPXHSjkO7bBc6MxYoE6JXF+0V1mIAwS+6siaCUOcD1NFCnmX6nQi5 XHrKRCyLWI+SZJdLTXq1AiTbPM72tfZ6Q3gnJF1Ieg99sU3eVLWTN724qhNNiAVFQ74p bSgwe/L71n1q+NsKdu/t0YmC4Z+QxjwnAHb7VtyDXwLD1FDOfGFlE9SVbr4vqY4rNpNN HlXA== MIME-Version: 1.0 X-Received: by 10.194.238.40 with SMTP id vh8mr18943056wjc.57.1416678173802; Sat, 22 Nov 2014 09:42:53 -0800 (PST) Sender: adamosdas@gmail.com X-Google-Sender-Delegation: adamosdas@gmail.com Received: by 10.216.177.198 with HTTP; Sat, 22 Nov 2014 09:42:53 -0800 (PST) In-Reply-To: References: <77A73EB3-7967-4AB4-B5E6-191B46898119@yahoo.it> <0B32BE05-B6B6-4FEF-B2E8-8B99A45F4A83@yahoo.it> <41384542-C90C-4573-992E-CDD5E29B5803@yahoo-inc.com> <536AC63F-4C87-4071-A1D9-D867DCC6769B@yahoo.it> <92F1CFC1-0961-49C3-AE1F-FE970BFA4405@yahoo.it> <89D518FA-00B2-40C4-B391-BDCFB6CA2B50@yahoo.it> Date: Sat, 22 Nov 2014 12:42:53 -0500 X-Google-Sender-Auth: Yiw7VPDtM8zNtKdZ_mVPXNQGAUc Message-ID: Subject: Re: Help with Giraph on Yarn From: D Adams To: user@giraph.apache.org Content-Type: multipart/alternative; boundary=089e01419f4e4f89250508761721 X-Virus-Checked: Checked by ClamAV on apache.org --089e01419f4e4f89250508761721 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I'm sorry, please excuse my ignorance, but, I have no idea where to find my configuration. That is, I have no idea where these lines came from (or what file): yarn 1029 0,1 1,2 3499572 209560 s001 S 10:29am 0:06.04 /Library/Java/ JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/bin/java -Dproc_resourcemanager -Xmx500m -Dhadoop.log.dir=3D/opt/yarn/hadoop-2.5.1/logs -Dyarn.log.dir=3D/opt/yarn/hadoop-2.5.1/logs ... As far as configuration goes, I'm only really aware of the etc/hadoop/*-env.sh and .xml files and main pom.xml file. Or is the above part of the command you use to run giraph jobs? I will make changes to hadoop-env.sh file as you suggest and let you know how it works out. On Fri, Nov 21, 2014 at 6:05 AM, Alessandro Negro wrote: > Hi, > I forgot to say that I added also this line: > > export HADOOP_CLASSPATH=3D$HADOOP_CLASSPATH:$MYLIB > > > Il giorno 21/nov/2014, alle ore 10:34, Alessandro Negro < > alenegro81@yahoo.it> ha scritto: > > Hi Das, > this is my configuration of hadoop: > > yarn 1029 0,1 1,2 3499572 209560 s001 S 10:29am > 0:06.04 > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/bin/java > -Dproc_resourcemanager -Xmx500m > -Dhadoop.log.dir=3D/opt/yarn/hadoop-2.5.1/logs > -Dyarn.log.dir=3D/opt/yarn/hadoop-2.5.1/logs > -Dhadoop.log.file=3Dyarn-yarn-resourcemanager-MacBook-Pro-di-Alessandro.l= ocal.log > -Dyarn.log.file=3Dyarn-yarn-resourcemanager-MacBook-Pro-di-Alessandro.loc= al.log > -Dyarn.home.dir=3D -Dyarn.id.str=3Dyarn -Dhadoop.root.logger=3DINFO,RFA > -Dyarn.root.logger=3DINFO,RFA > -Djava.library.path=3D/opt/yarn/hadoop-2.5.1/lib/native > -Dyarn.policy.file=3Dhadoop-policy.xml > -Dhadoop.log.dir=3D/opt/yarn/hadoop-2.5.1/logs > -Dyarn.log.dir=3D/opt/yarn/hadoop-2.5.1/logs > -Dhadoop.log.file=3Dyarn-yarn-resourcemanager-MacBook-Pro-di-Alessandro.l= ocal.log > -Dyarn.log.file=3Dyarn-yarn-resourcemanager-MacBook-Pro-di-Alessandro.loc= al.log > -Dyarn.home.dir=3D/opt/yarn/hadoop-2.5.1 > -Dhadoop.home.dir=3D/opt/yarn/hadoop-2.5.1 -Dhadoop.root.logger=3DINFO,RF= A > -Dyarn.root.logger=3DINFO,RFA > -Djava.library.path=3D/opt/yarn/hadoop-2.5.1/lib/native -classpath > /opt/yarn/hadoop-2.5.1/etc/hadoop:/opt/yarn/hadoop-2.5.1/etc/hadoop:/opt/= yarn/hadoop-2.5.1/etc/hadoop:/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib= /*:/opt/yarn/hadoop-2.5.1/share/hadoop/common/*:/opt/yarn/hadoop-2.5.1/shar= e/hadoop/hdfs:/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/*:/opt/yarn/hado= op-2.5.1/share/hadoop/hdfs/*:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib/*= :/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*:/opt/yarn/hadoop-2.5.1/share/ha= doop/mapreduce/lib/*:/opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/*::/opt/= yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/myGiraph/*.jar= :/Users/ale/extprj/giraph/myGiraph/lib/*.jar:/contrib/capacity-scheduler/*.= jar:/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/myGir= aph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar:/contrib/capacity-sch= eduler/*.jar:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*:/opt/yarn/hadoop-2.= 5.1/share/hadoop/yarn/lib/*:/opt/yarn/hadoop-2.5.1/etc/hadoop/rm-config/log= 4j.properties > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager > yarn 1073 0,1 1,0 3347132 174644 s001 S 10:29am > 0:04.77 > /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/bin/java > -Dproc_nodemanager -Xmx500m -Dhadoop.log.dir=3D/opt/yarn/hadoop-2.5.1/log= s > -Dyarn.log.dir=3D/opt/yarn/hadoop-2.5.1/logs > -Dhadoop.log.file=3Dyarn-yarn-nodemanager-MacBook-Pro-di-Alessandro.local= .log > -Dyarn.log.file=3Dyarn-yarn-nodemanager-MacBook-Pro-di-Alessandro.local.l= og > -Dyarn.home.dir=3D -Dyarn.id.str=3Dyarn -Dhadoop.root.logger=3DINFO,RFA > -Dyarn.root.logger=3DINFO,RFA > -Djava.library.path=3D/opt/yarn/hadoop-2.5.1/lib/native > -Dyarn.policy.file=3Dhadoop-policy.xml -server > -Dhadoop.log.dir=3D/opt/yarn/hadoop-2.5.1/logs > -Dyarn.log.dir=3D/opt/yarn/hadoop-2.5.1/logs > -Dhadoop.log.file=3Dyarn-yarn-nodemanager-MacBook-Pro-di-Alessandro.local= .log > -Dyarn.log.file=3Dyarn-yarn-nodemanager-MacBook-Pro-di-Alessandro.local.l= og > -Dyarn.home.dir=3D/opt/yarn/hadoop-2.5.1 > -Dhadoop.home.dir=3D/opt/yarn/hadoop-2.5.1 -Dhadoop.root.logger=3DINFO,RF= A > -Dyarn.root.logger=3DINFO,RFA > -Djava.library.path=3D/opt/yarn/hadoop-2.5.1/lib/native -classpath > /opt/yarn/hadoop-2.5.1/etc/hadoop:/opt/yarn/hadoop-2.5.1/etc/hadoop:/opt/= yarn/hadoop-2.5.1/etc/hadoop:/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib= /*:/opt/yarn/hadoop-2.5.1/share/hadoop/common/*:/opt/yarn/hadoop-2.5.1/shar= e/hadoop/hdfs:/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/*:/opt/yarn/hado= op-2.5.1/share/hadoop/hdfs/*:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib/*= :/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*:/opt/yarn/hadoop-2.5.1/share/ha= doop/mapreduce/lib/*:/opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/*::/opt/= yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/myGiraph/*.jar= :/Users/ale/extprj/giraph/myGiraph/lib/*.jar:/contrib/capacity-scheduler/*.= jar:/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/myGir= aph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar:/contrib/capacity-sch= eduler/*.jar:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*:/opt/yarn/hadoop-2.= 5.1/share/hadoop/yarn/lib/*:/opt/yarn/hadoop-2.5.1/etc/hadoop/nm-config/log= 4j.properties > org.apache.hadoop.yarn.server.nodemanager.NodeManager > > So I run it as yarn user in this way: > > sudo -u yarn =E2=80=A6. > > Moreover I have just notice that I added the following lines into the > hadoop-env.sh > > hadoop-env.sh:export > MYLIB=3D/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph= /myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar > > Accordingly to this other thread: > > > http://mail-archives.apache.org/mod_mbox/giraph-user/201408.mbox/%3C53F4C= 689.5060101@web.de%3E > > Let me know, > Alessandro > > > Il giorno 20/nov/2014, alle ore 23:51, D Adams ha > scritto: > > --f46d044283ecf837440508522939 > Content-Type: text/plain; charset=3DUTF-8 > Content-Transfer-Encoding: quoted-printable > > Ok, in the following, I use bin/yarn. Doesn't seem like much has changed: > > hduser@Roosevelt:/usr/local/hadoop$ ./hduser_jobs.sh > Running hduser script ... > bin/yarn jar > > /usr/local/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-f= or=3D > -hadoop-2.5.1-jar-with-dependencies.jar > org.apache.giraph.GiraphRunner > org.apache.giraph.examples.SimpleShortestPathsComputation -vif > org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat > -vip /user/hduser/input/tiny_graph.txt -vof > org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op > /user/hduser/output/shortestpaths -w 1 -yj > giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar > 14/11/18 05:36:05 WARN util.NativeCodeLoader: Unable to load native-hadoo= p > library for your platform... using builtin-java classes where applicable > 14/11/18 05:36:08 INFO utils.ConfigurationUtils: No edge input format > specified. Ensure your InputFormat does not require one. > 14/11/18 05:36:08 INFO utils.ConfigurationUtils: No edge output format > specified. Ensure your OutputFormat does not require one. > 14/11/18 05:36:08 INFO yarn.GiraphYarnClient: Final output path is: > hdfs://localhost:9000/user/hduser/output/shortestpaths > 14/11/18 05:36:08 INFO yarn.GiraphYarnClient: Running Client > 14/11/18 05:36:08 INFO client.RMProxy: Connecting to ResourceManager at > localhost/127.0.0.1:8050 > 14/11/18 05:36:09 INFO yarn.GiraphYarnClient: Got node report from ASM fo= r, > nodeId=3D3Droosevelt:60486, nodeAddress roosevelt:8042, nodeRackName > /default-rack, nodeNumContainers 0 > 14/11/18 05:36:09 INFO yarn.GiraphYarnClient: Defaulting per-task heap si= ze > to 1024MB. > 14/11/18 05:36:09 INFO yarn.GiraphYarnClient: Obtained new Application ID= : > application_1416310496070_0001 > 14/11/18 05:36:09 INFO Configuration.deprecation: mapred.job.id is > deprecated. Instead, use mapreduce.job.id > 14/11/18 05:36:09 INFO yarn.GiraphYarnClient: Set the environment for the > application master > 14/11/18 05:36:09 INFO yarn.GiraphYarnClient: Environment for AM > > :{CLASSPATH=3D3D${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/sha= re/h=3D > > adoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS= _H=3D > > OME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP= _Y=3D > > ARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$H= AD=3D > > OOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop= /m=3D > apreduce/lib/*} > 14/11/18 05:36:09 INFO yarn.GiraphYarnClient: buildLocalResourceMap .... > 14/11/18 05:36:12 INFO yarn.YarnUtils: Registered file in LocalResources = :: > > hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application_14163= 10=3D > 496070_0001/giraph-conf.xml > 14/11/18 05:36:12 INFO yarn.GiraphYarnClient: LIB JARS > :giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.ja= r > 14/11/18 05:36:12 INFO yarn.YarnUtils: Class path name . > 14/11/18 05:36:12 INFO yarn.YarnUtils: base path checking . > 14/11/18 05:36:12 INFO yarn.GiraphYarnClient: Made local resource for > > :/usr/local/hadoop/share/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-= ja=3D > r-with-dependencies.jar > to > > hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application_14163= 10=3D > > 496070_0001/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-depe= nd=3D > encies.jar > 14/11/18 05:36:16 INFO yarn.YarnUtils: Registered file in LocalResources = :: > > hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application_14163= 10=3D > > 496070_0001/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-depe= nd=3D > encies.jar > 14/11/18 05:36:16 INFO yarn.GiraphYarnClient: ApplicationSumbissionContex= t > for GiraphApplicationMaster launch container is populated. > 14/11/18 05:36:16 INFO yarn.GiraphYarnClient: Submitting application to A= SM > 14/11/18 05:36:16 INFO impl.YarnClientImpl: Submitted application > application_1416310496070_0001 > 14/11/18 05:36:16 INFO yarn.GiraphYarnClient: Got new appId after > submission :application_1416310496070_0001 > 14/11/18 05:36:16 INFO yarn.GiraphYarnClient: GiraphApplicationMaster > container request was submitted to ResourceManager for job: Giraph: > org.apache.giraph.examples.SimpleShortestPathsComputation > 14/11/18 05:36:17 INFO yarn.GiraphYarnClient: Giraph: > org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 1.01 > secs > 14/11/18 05:36:17 INFO yarn.GiraphYarnClient: > appattempt_1416310496070_0001_000001, State: ACCEPTED, Containers used: 1 > 14/11/18 05:36:22 INFO yarn.GiraphYarnClient: Giraph: > org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 5.86 > secs > 14/11/18 05:36:22 INFO yarn.GiraphYarnClient: > appattempt_1416310496070_0001_000001, State: ACCEPTED, Containers used: 1 > 14/11/18 05:36:26 INFO yarn.GiraphYarnClient: Giraph: > org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 10.02 > secs > 14/11/18 05:36:26 INFO yarn.GiraphYarnClient: > appattempt_1416310496070_0001_000001, State: ACCEPTED, Containers used: 1 > 14/11/18 05:36:30 INFO yarn.GiraphYarnClient: Giraph: > org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 14.39 > secs > 14/11/18 05:36:30 INFO yarn.GiraphYarnClient: > appattempt_1416310496070_0001_000001, State: ACCEPTED, Containers used: 1 > 14/11/18 05:36:34 INFO yarn.GiraphYarnClient: Giraph: > org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 18.57 > secs > 14/11/18 05:36:34 INFO yarn.GiraphYarnClient: > appattempt_1416310496070_0001_000001, State: RUNNING, Containers used: 2 > > gam_stderr.log > > LF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > > [jar:file:/app/hadoop/tmp/nm-local-dir/usercache/hduser/appcache/applicat= io=3D > > n_1416310496070_0001/filecache/10/giraph-examples-1.1.0-SNAPSHOT-for-hado= op=3D > -2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class= ] > SLF4J: Found binding in > > [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.j= ar=3D > !/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explana=3D > tion. > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > > gam_stdout.log > > 2014-11-18 05:36:28,143 INFO [main] yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:main(421)) - Starting GitaphAM > 2014-11-18 05:36:30,404 WARN [main] util.NativeCodeLoader > (NativeCodeLoader.java:(62)) - Unable to load native-hadoop > library for your platform... using builtin-java classes where > applicable > 2014-11-18 05:36:32,649 INFO [main] yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:(168)) - GiraphAM for ContainerId > container_1416310496070_0001_01_000001 ApplicationAttemptId > appattempt_1416310496070_0001_000001 > 2014-11-18 05:36:32,852 INFO [main] client.RMProxy > (RMProxy.java:createRMProxy(92)) - Connecting to ResourceManager at > localhost/127.0.0.1:8030 > 2014-11-18 05:36:32,888 INFO [main] impl.NMClientAsyncImpl > (NMClientAsyncImpl.java:serviceInit(107)) - Upper bound of the thread > pool size is 500 > 2014-11-18 05:36:32,889 INFO [main] > impl.ContainerManagementProtocolProxy > (ContainerManagementProtocolProxy.java:(78)) - > yarn.client.max-nodemanagers-proxies : 500 > 2014-11-18 05:36:33,589 INFO [main] yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:setupContainerAskForRM(279)) - Requested > container ask: Capability[]Priority[10] > 2014-11-18 05:36:33,617 INFO [main] yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:setupContainerAskForRM(279)) - Requested > container ask: Capability[]Priority[10] > 2014-11-18 05:36:33,617 INFO [main] yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:run(185)) - Wait to finish .. > 2014-11-18 05:36:35,674 INFO [AMRM Heartbeater thread] > impl.AMRMClientImpl (AMRMClientImpl.java:populateNMTokens(299)) - > Received new token for : roosevelt:60486 > 2014-11-18 05:36:35,681 INFO [AMRM Callback Handler Thread] > yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:onContainersAllocated(605)) - Got > response from RM for container ask, allocatedCnt=3D3D1 > 2014-11-18 05:36:35,681 INFO [AMRM Callback Handler Thread] > yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:onContainersAllocated(608)) - Total > allocated # of container so far : 1 allocated out of 2 required. > 2014-11-18 05:36:35,681 INFO [AMRM Callback Handler Thread] > yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:startContainerLaunchingThreads(359)) - > Launching command on a new container., > containerId=3D3Dcontainer_1416310496070_0001_01_000002, > containerNode=3D3Droosevelt:60486, containerNodeURI=3D3Droosevelt:8042, > containerResourceMemory=3D3D1024 > 2014-11-18 05:36:35,688 INFO [pool-4-thread-1] > yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:buildContainerLaunchContext(492)) - > Setting up container launch container for > containerid=3D3Dcontainer_1416310496070_0001_01_000002 > 2014-11-18 05:36:35,720 INFO [pool-4-thread-1] > yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:buildContainerLaunchContext(498)) - > Conatain launch Commands :java -Xmx1024M -Xms1024M -cp .:${CLASSPATH} > org.apache.giraph.yarn.GiraphYarnTask 1416310496070 1 2 1 > 1>/task-2-stdout.log 2>/task-2-stderr.log > 2014-11-18 05:36:35,721 INFO [pool-4-thread-1] > yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:buildContainerLaunchContext(518)) - > Setting username in ContainerLaunchContext to: hduser > 2014-11-18 05:36:36,789 INFO [AMRM Callback Handler Thread] > yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:onContainersAllocated(605)) - Got > response from RM for container ask, allocatedCnt=3D3D1 > 2014-11-18 05:36:36,789 INFO [AMRM Callback Handler Thread] > yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:onContainersAllocated(608)) - Total > allocated # of container so far : 2 allocated out of 2 required. > 2014-11-18 05:36:36,790 INFO [AMRM Callback Handler Thread] > yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:startContainerLaunchingThreads(359)) - > Launching command on a new container., > containerId=3D3Dcontainer_1416310496070_0001_01_000003, > containerNode=3D3Droosevelt:60486, containerNodeURI=3D3Droosevelt:8042, > containerResourceMemory=3D3D1024 > 2014-11-18 05:36:37,227 INFO [pool-4-thread-2] > yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:buildContainerLaunchContext(492)) - > Setting up container launch container for > containerid=3D3Dcontainer_1416310496070_0001_01_000003 > 2014-11-18 05:36:37,227 INFO [pool-4-thread-2] > yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:buildContainerLaunchContext(498)) - > Conatain launch Commands :java -Xmx1024M -Xms1024M -cp .:${CLASSPATH} > org.apache.giraph.yarn.GiraphYarnTask 1416310496070 1 3 1 > 1>/task-3-stdout.log 2>/task-3-stderr.log > 2014-11-18 05:36:37,227 INFO [pool-4-thread-2] > yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:buildContainerLaunchContext(518)) - > Setting username in ContainerLaunchContext to: hduser > 2014-11-18 05:36:37,463 INFO [pool-4-thread-1] yarn.YarnUtils > (YarnUtils.java:addFsResourcesToMap(72)) - Adding > giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar > to LocalResources for export.to > > hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application_14163= 10=3D > > 496070_0001/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-depe= nd=3D > encies.jar > 2014-11-18 05:36:37,678 INFO [pool-4-thread-1] yarn.YarnUtils > (YarnUtils.java:addFileToResourceMap(160)) - Registered file in > LocalResources :: > > hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application_14163= 10=3D > > 496070_0001/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-depe= nd=3D > encies.jar > 2014-11-18 05:36:37,678 WARN [pool-4-thread-1] yarn.YarnUtils > (YarnUtils.java:addFsResourcesToMap(81)) - Job jars (-yj option) > didn't include giraph-core. > 2014-11-18 05:36:37,680 INFO [pool-4-thread-1] yarn.YarnUtils > (YarnUtils.java:addFileToResourceMap(160)) - Registered file in > LocalResources :: > > hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application_14163= 10=3D > 496070_0001/giraph-conf.xml > 2014-11-18 05:36:37,694 INFO > [org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl #0] > impl.NMClientAsyncImpl (NMClientAsyncImpl.java:run(531)) - Processing > Event EventType: START_CONTAINER for Container > container_1416310496070_0001_01_000002 > 2014-11-18 05:36:37,699 INFO > [org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl #1] > impl.NMClientAsyncImpl (NMClientAsyncImpl.java:run(531)) - Processing > Event EventType: START_CONTAINER for Container > container_1416310496070_0001_01_000003 > 2014-11-18 05:36:37,698 INFO > [org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl #0] > impl.ContainerManagementProtocolProxy > (ContainerManagementProtocolProxy.java:newProxy(212)) - Opening proxy > : roosevelt:60486 > 2014-11-18 05:36:37,761 INFO > [org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl #2] > impl.NMClientAsyncImpl (NMClientAsyncImpl.java:run(531)) - Processing > Event EventType: QUERY_CONTAINER for Container > container_1416310496070_0001_01_000003 > 2014-11-18 05:36:37,771 INFO > [org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl #3] > impl.NMClientAsyncImpl (NMClientAsyncImpl.java:run(531)) - Processing > Event EventType: QUERY_CONTAINER for Container > container_1416310496070_0001_01_000002 > > > > On Thu, Nov 20, 2014 at 1:32 PM, Alessandro Negro > wrote: > > Hi Das, > I mean as user you run Hadoop yarn resource manager. > > Alessandro > > Il giorno 20/nov/2014, alle ore 19:22, D Adams ha > scritto: > > Alessandro, > I'm not sure what you mean, should I create a new user on my system > named 'yarn'? I'm new to both hadoop and giraph, so I'm not sure. > > Tripti, > I'll get those logs as soon as I can. > > Thank you both, > > V/r > Das > On Nov 20, 2014 3:13 AM, "Alessandro Negro" wrote: > > Hi Tripti, > I agree that a more detailed error log could be useful. > > Thanks, > Alessandro > > > > > Il giorno 19/nov/2014, alle ore 15:54, Tripti Singh > m> > > ha scritto: > > --_000_D092AB9C1CB42triptiyahooinccom_ > Content-Type: text/plain; charset=3D3D"Windows-1252" > Content-Transfer-Encoding: quoted-printable > > Hi Alessandro, > > I guess u r talking about the documentation on Giraph webpage which > > mention=3D3D > > s addition of this new option which is more or less mandatory for yarn > > base=3D3D > > d profiles. > When I first ran the Giraph on yarn without the proper =3D3D96yj option, > > there =3D3D > > was no way I could figure out from the Application/Container logs that > > this=3D3D > > was the issue. > I think, it=3D3D92ll be a good idea to have this message in the logs for > > easy d=3D3D > > ebugging. > > Thanks, > Tripti. > > > From: Alessandro Negro > > > Reply-To: "user@giraph.apache.org >" > > > aph.apache.org>> > Date: Tuesday, November 18, 2014 at 4:31 PM > To: "user@giraph.apache.org >" < > > user@giraph.ap=3D3D > > ache.org>> > Subject: Re: Help with Giraph on Yarn > > Hi Eli, > no I think that the help message is clear enough, generally when I rea=3D > > d > > =3D3D93=3D3D > > jar=3D3D94 I always mean absolute path for jar, but this is a my persona= =3D > > l > > misun=3D3D > > derstanding. > > Thanks, > Alessandro > > > Il giorno 16/nov/2014, alle ore 21:39, Eli Reisman < > > apache.mailbox@gmail.co=3D3D > > m>> ha scritto= : > > Sounds like you got it figured out since last time I checked this list=3D > > , > > sor=3D3D > > ry it was a pain. Feel free to drop a JIRA ticket if we can make help > > messa=3D3D > > ge etc. for -yj option more clear, there's lots to improve there. > > On Sat, Nov 8, 2014 at 7:26 AM, Alessandro Negro > > o:alenegro81@yahoo.it>> wrote: > Hi Tripti, > finally I was able to run the test with success. It was an issue of > > permiss=3D3D > > ion since I was running as ale not as yarn. > Let me say that now I=3D3D92m able to run graph examples on Yarn 2.5.1. > > This is=3D3D > > the final result: > > 14/11/08 16:24:00 INFO yarn.GiraphYarnClient: Completed Giraph: > > org.apache.=3D3D > > giraph.examples.SimpleShortestPathsComputation: SUCCEEDED, total > > running ti=3D3D > > me: 0 minutes, 21 seconds. > > > Many thanks for your support, > Alessandro > > Il giorno 06/nov/2014, alle ore 15:16, Tripti Singh < > > tripti@yahoo-inc.com > ailto:tripti@yahoo-inc.com>> ha scritto: > > I don't know if u have access to this node. But if u do, u could check > > if t=3D3D > > he file is indeed there and u have access to it. > > Sent from my iPhone > > On 06-Nov-2014, at 6:12 pm, "Alessandro Negro" > > alenegro81@yahoo.it>> wrote: > > You are right it works, but now I receive the following error: > > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > > [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/use=3D3D > > > rcache/ale/appcache/application_1415264041937_0009/filecache/10/giraph-e= =3D > > xam=3D3D > > > ples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4= =3D > > j/i=3D3D > > mpl/StaticLoggerBinder.class] > SLF4J: Found binding in > > [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/commo=3D3D > > n/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class=3D > > ] > > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > > explana=3D3D > > tion. > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > 2014-11-06 13:15:37.120 java[10158:1803] Unable to load realm info fro=3D > > m > > SCD=3D3D > > ynamicStore > Exception in thread "pool-4-thread-1" java.lang.IllegalStateException: > > Coul=3D3D > > d not configure the containerlaunch context for GiraphYarnTasks. > at > > org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(Giraph= =3D > > =3D3D > > ApplicationMaster.java:391) > at > > org.apache.giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicat= =3D > > =3D3D > > ionMaster.java:78) > at > > org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.b= =3D > > =3D3D > > uildContainerLaunchContext(GiraphApplicationMaster.java:522) > at > > org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.r= =3D > > =3D3D > > un(GiraphApplicationMaster.java:479) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav= =3D > > =3D3D > > a:1145) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja= =3D > > =3D3D > > va:615) > at java.lang.Thread.run(Thread.java:744) > Caused by: java.io.FileNotFoundException: File does not exist: > > hdfs://hadoo=3D3D > > > p-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_= =3D > > 000=3D3D > > > 9/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.= =3D > > jar > > at > > org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSy= =3D > > =3D3D > > stem.java:1072) > at > > org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSy= =3D > > =3D3D > > stem.java:1064) > at > > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolv= =3D > > =3D3D > > er.java:81) > at > > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFi= =3D > > =3D3D > > leSystem.java:1064) > at > > org.apache.giraph.yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153= =3D > > =3D3D > > ) > at > > org.apache.giraph.yarn.YarnUtils.addFsResourcesToMap(YarnUtils.java:77) > > at > > org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(Giraph= =3D > > =3D3D > > ApplicationMaster.java:387) > ... 6 more > > > That justify the other error I receive in the task: > Could not find or load main class org.apache.giraph.yarn.GiraphYarnTas=3D > > k > > > Thanks, > > Il giorno 06/nov/2014, alle ore 13:07, Tripti Singh < > > tripti@yahoo-inc.com > ailto:tripti@yahoo-inc.com>> ha scritto: > > Why r u adding two jars? Example jar ideally contains core library so > > every=3D3D > > thing should be available with just one example jar included > > Sent from my iPhone > > On 06-Nov-2014, at 4:33 pm, "Alessandro Negro" > > alenegro81@yahoo.it>> wrote: > > Hi, > now it seems better, I need to add: > > > giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,giraph-= =3D > > exa=3D3D > > mples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar > > > Now it seems that after a lot of cycle it fail with this error: > > Could not find or load main class org.apache.giraph.yarn.GiraphYarnTas=3D > > k > > > But in this case the error appear in task-3-stderr.log not in > > gam-stderr.l=3D3D > > og where there is the following error: > > LF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > > [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/use=3D3D > > > rcache/ale/appcache/application_1415264041937_0006/filecache/12/giraph-1= =3D > > .1.=3D3D > > > 0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/St= =3D > > ati=3D3D > > cLoggerBinder.class] > SLF4J: Found binding in > > [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/use=3D3D > > > rcache/ale/appcache/application_1415264041937_0006/filecache/10/giraph-e= =3D > > xam=3D3D > > > ples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4= =3D > > j/i=3D3D > > mpl/StaticLoggerBinder.class] > SLF4J: Found binding in > > [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/commo=3D3D > > n/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class=3D > > ] > > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > > > > --089e01419f4e4f89250508761721 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I'm sorry, please excuse my ignorance, but, I hav= e no idea where to find my configuration. That is, I have no idea where the= se lines came from (or what file):

yarn =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 1029 =C2=A0 0,1=C2=A0 1,2=C2=A0 3499572 209560 s001=C2=A0 S= =C2=A0 =C2=A0 10:29am =C2=A0 0:06.04 /Library/Java/
JavaVirtualMachines= /jdk1.7.0_45.jdk/Contents/Home/bin/java -Dproc_resourcemanager -Xmx500m -Dh= adoop.log.dir=3D/opt/yarn/hadoop-2.5.1/logs -Dyarn.log.dir=3D/opt/yarn/hado= op-2.5.1/logs
...
As far as configuration goes, I'm= only really aware of the etc/hadoop/*-env.sh and .xml files and main pom.x= ml file. Or is the above part of the command you use to run giraph jobs?
I will make changes to hadoop-env.sh file as you suggest and le= t you know how it works out.

=
On Fri, Nov 21, 2014 at 6:05 AM, Alessandro Negr= o <alenegro81@yahoo.it> wrote:
Hi,
I forgot to say that I a= dded also this line:=C2=A0

export HADOOP_CLASSPATH=3D$HADOOP_CLASS= PATH:$MYLIB


<= /div>
Il giorno 21/nov/2014, alle ore 10:34, Alessandro Negro <= alenegro81@yahoo.i= t> ha scritto:

Hi Das,
this is my configurati= on of hadoop:

yarn =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1029 = =C2=A0 0,1=C2=A0 1,2=C2=A0 3499572 209560 s001=C2=A0 S=C2=A0 =C2=A0 10:29am= =C2=A0 0:06.04 /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/= Home/bin/java -Dproc_resourcemanager -Xmx500m -Dhadoop.log.dir=3D/opt/yarn/= hadoop-2.5.1/logs -Dyarn.log.dir=3D/opt/yarn/hadoop-2.5.1/logs -Dhadoop.log= .file=3Dyarn-yarn-resourcemanager-MacBook-Pro-di-Alessandro.local.log -Dyar= n.log.file=3Dyarn-yarn-resourcemanager-MacBook-Pro-di-Alessandro.local.log = -Dyarn.home.dir=3D -Dyarn.id.str=3Dyarn -Dhadoop.root.logger=3DINFO,RFA -Dy= arn.root.logger=3DINFO,RFA -Djava.library.path=3D/opt/yarn/hadoop-2.5.1/lib= /native -Dyarn.policy.file=3Dhadoop-policy.xml -Dhadoop.log.dir=3D/opt/yarn= /hadoop-2.5.1/logs -Dyarn.log.dir=3D/opt/yarn/hadoop-2.5.1/logs -Dhadoop.lo= g.file=3Dyarn-yarn-resourcemanager-MacBook-Pro-di-Alessandro.local.log -Dya= rn.log.file=3Dyarn-yarn-resourcemanager-MacBook-Pro-di-Alessandro.local.log= -Dyarn.home.dir=3D/opt/yarn/hadoop-2.5.1 -Dhadoop.home.dir=3D/opt/yarn/had= oop-2.5.1 -Dhadoop.root.logger=3DINFO,RFA -Dyarn.root.logger=3DINFO,RFA -Dj= ava.library.path=3D/opt/yarn/hadoop-2.5.1/lib/native -classpath /opt/yarn/h= adoop-2.5.1/etc/hadoop:/opt/yarn/hadoop-2.5.1/etc/hadoop:/opt/yarn/hadoop-2= .5.1/etc/hadoop:/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/*:/opt/yarn/= hadoop-2.5.1/share/hadoop/common/*:/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs= :/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/*:/opt/yarn/hadoop-2.5.1/shar= e/hadoop/hdfs/*:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib/*:/opt/yarn/ha= doop-2.5.1/share/hadoop/yarn/*:/opt/yarn/hadoop-2.5.1/share/hadoop/mapreduc= e/lib/*:/opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/*::/opt/yarn/hadoop-2= .5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/myGiraph/*.jar:/Users/ale/e= xtprj/giraph/myGiraph/lib/*.jar:/contrib/capacity-scheduler/*.jar:/opt/yarn= /hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/myGiraph/*.jar:/Us= ers/ale/extprj/giraph/myGiraph/lib/*.jar:/contrib/capacity-scheduler/*.jar:= /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*:/opt/yarn/hadoop-2.5.1/share/had= oop/yarn/lib/*:/opt/yarn/hadoop-2.5.1/etc/hadoop/rm-config/log4j.properties= org.apache.hadoop.yarn.server.resourcemanager.ResourceManager
yarn =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 1073 =C2=A0 0,1=C2=A0 1,0=C2=A0 3347132 174644 s00= 1=C2=A0 S=C2=A0 =C2=A0 10:29am =C2=A0 0:04.77 /Library/Java/JavaVirtualMach= ines/jdk1.7.0_45.jdk/Contents/Home/bin/java -Dproc_nodemanager -Xmx500m -Dh= adoop.log.dir=3D/opt/yarn/hadoop-2.5.1/logs -Dyarn.log.dir=3D/opt/yarn/hado= op-2.5.1/logs -Dhadoop.log.file=3Dyarn-yarn-nodemanager-MacBook-Pro-di-Ales= sandro.local.log -Dyarn.log.file=3Dyarn-yarn-nodemanager-MacBook-Pro-di-Ale= ssandro.local.log -Dyarn.home.dir=3D -Dyarn.id.str=3Dyarn -Dhadoop.root.log= ger=3DINFO,RFA -Dyarn.root.logger=3DINFO,RFA -Djava.library.path=3D/opt/yar= n/hadoop-2.5.1/lib/native -Dyarn.policy.file=3Dhadoop-policy.xml -server -D= hadoop.log.dir=3D/opt/yarn/hadoop-2.5.1/logs -Dyarn.log.dir=3D/opt/yarn/had= oop-2.5.1/logs -Dhadoop.log.file=3Dyarn-yarn-nodemanager-MacBook-Pro-di-Ale= ssandro.local.log -Dyarn.log.file=3Dyarn-yarn-nodemanager-MacBook-Pro-di-Al= essandro.local.log -Dyarn.home.dir=3D/opt/yarn/hadoop-2.5.1 -Dhadoop.home.d= ir=3D/opt/yarn/hadoop-2.5.1 -Dhadoop.root.logger=3DINFO,RFA -Dyarn.root.log= ger=3DINFO,RFA -Djava.library.path=3D/opt/yarn/hadoop-2.5.1/lib/native -cla= sspath /opt/yarn/hadoop-2.5.1/etc/hadoop:/opt/yarn/hadoop-2.5.1/etc/hadoop:= /opt/yarn/hadoop-2.5.1/etc/hadoop:/opt/yarn/hadoop-2.5.1/share/hadoop/commo= n/lib/*:/opt/yarn/hadoop-2.5.1/share/hadoop/common/*:/opt/yarn/hadoop-2.5.1= /share/hadoop/hdfs:/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/*:/opt/yarn= /hadoop-2.5.1/share/hadoop/hdfs/*:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/= lib/*:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*:/opt/yarn/hadoop-2.5.1/sha= re/hadoop/mapreduce/lib/*:/opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/*::= /opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/myGiraph/= *.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar:/contrib/capacity-schedul= er/*.jar:/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/= myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar:/contrib/capacit= y-scheduler/*.jar:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*:/opt/yarn/hado= op-2.5.1/share/hadoop/yarn/lib/*:/opt/yarn/hadoop-2.5.1/etc/hadoop/nm-confi= g/log4j.properties org.apache.hadoop.yarn.server.nodemanager.NodeManager

So I run it as yarn user in this way:

=
sudo -u yarn =E2=80=A6.

Moreover I have= just notice that I added the following lines into the hadoop-env.sh
<= div>
hadoop-env.sh:export MYLIB=3D/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/U= sers/ale/extprj/giraph/myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib= /*.jar

Accordingly to this other thread:


Let me know,
Alessa= ndro


Il giorno 20/nov/2014, alle ore= 23:51, D Adams <dadamszx@gmail.com> ha scritto:

=
--f46d04= 4283ecf837440508522939
Content-Type: text/plain; charset=3DUTF-8
Cont= ent-Transfer-Encoding: quoted-printable

Ok, in the following, I use = bin/yarn. Doesn't seem like much has changed:

hduser@Roosevelt:/= usr/local/hadoop$ ./hduser_jobs.sh
Running hduser script ...
bin/yarn= jar
/usr/local/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAP= SHOT-for=3D
-hadoop-2.5.1-jar-with-dependencies.jar
org.apache.giraph= .GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation = -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFo= rmat
-vip /user/hduser/input/tiny_graph.txt -vof
org.apache.giraph.io= .formats.IdWithValueTextOutputFormat -op
/user/hduser/output/shortestpat= hs -w 1 -yj
giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dep= endencies.jar
14/11/18 05:36:05 WARN util.NativeCodeLoader: Unable to lo= ad native-hadoop
library for your platform... using builtin-java classes= where applicable
14/11/18 05:36:08 INFO utils.ConfigurationUtils: No ed= ge input format
specified. Ensure your InputFormat does not require one.=
14/11/18 05:36:08 INFO utils.ConfigurationUtils: No edge output format<= br>specified. Ensure your OutputFormat does not require one.
14/11/18 05= :36:08 INFO yarn.GiraphYarnClient: Final output path is:
hdfs://local= host:9000/user/hduser/output/shortestpaths
14/11/18 05:36:08 INFO ya= rn.GiraphYarnClient: Running Client
14/11/18 05:36:08 INFO client.RMProx= y: Connecting to ResourceManager at
localhost/127.0.0.1:8050
14/11/18 05:36:09 INFO yarn= .GiraphYarnClient: Got node report from ASM for,
nodeId=3D3Droosevelt:60= 486, nodeAddress roosevelt:8042, nodeRackName
/default-rack, nodeNumCont= ainers 0
14/11/18 05:36:09 INFO yarn.GiraphYarnClient: Defaulting per-ta= sk heap size
to 1024MB.
14/11/18 05:36:09 INFO yarn.GiraphYarnClient:= Obtained new Application ID:
application_1416310496070_0001
14/11/18= 05:36:09 INFO Configuration.deprecation: mapred.job.id is
deprecated. Instead, use mapreduce.job.id
14/1= 1/18 05:36:09 INFO yarn.GiraphYarnClient: Set the environment for the
ap= plication master
14/11/18 05:36:09 INFO yarn.GiraphYarnClient: Environme= nt for AM
:{CLASSPATH=3D3D${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMM= ON_HOME/share/h=3D
adoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/commo= n/lib/*:$HADOOP_HDFS_H=3D
OME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/shar= e/hadoop/hdfs/lib/*:$HADOOP_Y=3D
ARN_HOME/share/hadoop/yarn/*:$HADOOP_YA= RN_HOME/share/hadoop/yarn/lib/*:$HAD=3D
OOP_MAPRED_HOME/share/hadoop/map= reduce/*:$HADOOP_MAPRED_HOME/share/hadoop/m=3D
apreduce/lib/*}
14/11/= 18 05:36:09 INFO yarn.GiraphYarnClient: buildLocalResourceMap ....
14/11= /18 05:36:12 INFO yarn.YarnUtils: Registered file in LocalResources ::
<= a>hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application_14163= 10=3D
496070_0001/giraph-conf.xml
14/11/18 05:36:12 INFO yarn.Gir= aphYarnClient: LIB JARS
:giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1= -jar-with-dependencies.jar
14/11/18 05:36:12 INFO yarn.YarnUtils: Class = path name .
14/11/18 05:36:12 INFO yarn.YarnUtils: base path checking .<= br>14/11/18 05:36:12 INFO yarn.GiraphYarnClient: Made local resource for:/usr/local/hadoop/share/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-j= a=3D
r-with-dependencies.jar
to
hdfs://localhost:9000/user/hdus= er/giraph_yarn_jar_cache/application_1416310=3D
496070_0001/giraph-e= xamples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-depend=3D
encies.jar14/11/18 05:36:16 INFO yarn.YarnUtils: Registered file in LocalResources := :
hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application= _1416310=3D
496070_0001/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.= 5.1-jar-with-depend=3D
encies.jar
14/11/18 05:36:16 INFO yarn.GiraphY= arnClient: ApplicationSumbissionContext
for GiraphApplicationMaster laun= ch container is populated.
14/11/18 05:36:16 INFO yarn.GiraphYarnClient:= Submitting application to ASM
14/11/18 05:36:16 INFO impl.YarnClientImp= l: Submitted application
application_1416310496070_0001
14/11/18 05:3= 6:16 INFO yarn.GiraphYarnClient: Got new appId after
submission :applica= tion_1416310496070_0001
14/11/18 05:36:16 INFO yarn.GiraphYarnClient: Gi= raphApplicationMaster
container request was submitted to ResourceManager= for job: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputat= ion
14/11/18 05:36:17 INFO yarn.GiraphYarnClient: Giraph:
org.apache.= giraph.examples.SimpleShortestPathsComputation, Elapsed: 1.01
secs
14= /11/18 05:36:17 INFO yarn.GiraphYarnClient:
appattempt_1416310496070_000= 1_000001, State: ACCEPTED, Containers used: 1
14/11/18 05:36:22 INFO yar= n.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPat= hsComputation, Elapsed: 5.86
secs
14/11/18 05:36:22 INFO yarn.GiraphY= arnClient:
appattempt_1416310496070_0001_000001, State: ACCEPTED, Contai= ners used: 1
14/11/18 05:36:26 INFO yarn.GiraphYarnClient: Giraph:
or= g.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 10.02
= secs
14/11/18 05:36:26 INFO yarn.GiraphYarnClient:
appattempt_1416310= 496070_0001_000001, State: ACCEPTED, Containers used: 1
14/11/18 05:36:3= 0 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleS= hortestPathsComputation, Elapsed: 14.39
secs
14/11/18 05:36:30 INFO y= arn.GiraphYarnClient:
appattempt_1416310496070_0001_000001, State: ACCEP= TED, Containers used: 1
14/11/18 05:36:34 INFO yarn.GiraphYarnClient: Gi= raph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed= : 18.57
secs
14/11/18 05:36:34 INFO yarn.GiraphYarnClient:
appatte= mpt_1416310496070_0001_000001, State: RUNNING, Containers used: 2

ga= m_stderr.log

LF4J: Class path contains multiple SLF4J bindings.
S= LF4J: Found binding in
[jar:file:/app/hadoop/tmp/nm-local-dir/usercache/= hduser/appcache/applicatio=3D
n_1416310496070_0001/filecache/10/giraph-e= xamples-1.1.0-SNAPSHOT-for-hadoop=3D
-2.5.1-jar-with-dependencies.jar!/o= rg/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:= file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar=3D!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See=C2=A0http://www.slf4j.org/codes.html#multiple_bindings=C2=A0for an explana=3D
tion.
SLF4J: Actual binding is of type [org.slf= 4j.impl.Log4jLoggerFactory]

gam_stdout.log

2014-11-18 05:36:2= 8,143 INFO =C2=A0[main] yarn.GiraphApplicationMaster
(GiraphApplicationM= aster.java:main(421)) - Starting GitaphAM
2014-11-18 05:36:30,404 WARN = =C2=A0[main] util.NativeCodeLoader
(NativeCodeLoader.java:<clinit>= (62)) - Unable to load native-hadoop
library for your platform... using = builtin-java classes where
applicable
2014-11-18 05:36:32,649 INFO = =C2=A0[main] yarn.GiraphApplicationMaster
(GiraphApplicationMaster.java:= <init>(168)) - GiraphAM =C2=A0for ContainerId
container_1416310496= 070_0001_01_000001 ApplicationAttemptId
appattempt_1416310496070_0001_00= 0001
2014-11-18 05:36:32,852 INFO =C2=A0[main] client.RMProxy
(RMProx= y.java:createRMProxy(92)) - Connecting to ResourceManager at
localhost/<= a href=3D"http://127.0.0.1:8030" target=3D"_blank">127.0.0.1:8030
20= 14-11-18 05:36:32,888 INFO =C2=A0[main] impl.NMClientAsyncImpl
(NMClient= AsyncImpl.java:serviceInit(107)) - Upper bound of the thread
pool size i= s 500
2014-11-18 05:36:32,889 INFO =C2=A0[main]
impl.ContainerManagem= entProtocolProxy
(ContainerManagementProtocolProxy.java:<init>(78)= ) -
yarn.client.max-nodemanagers-proxies : 500
2014-11-18 05:36:33,58= 9 INFO =C2=A0[main] yarn.GiraphApplicationMaster
(GiraphApplicationMaste= r.java:setupContainerAskForRM(279)) - Requested
container ask: Capabilit= y[<memory:1024, vCores:0>]Priority[10]
2014-11-18 05:36:33,617 INF= O =C2=A0[main] yarn.GiraphApplicationMaster
(GiraphApplicationMaster.jav= a:setupContainerAskForRM(279)) - Requested
container ask: Capability[<= ;memory:1024, vCores:0>]Priority[10]
2014-11-18 05:36:33,617 INFO =C2= =A0[main] yarn.GiraphApplicationMaster
(GiraphApplicationMaster.java:run= (185)) - Wait to finish ..
2014-11-18 05:36:35,674 INFO =C2=A0[AMRM Hear= tbeater thread]
impl.AMRMClientImpl (AMRMClientImpl.java:populateNMToken= s(299)) -
Received new token for : roosevelt:60486
2014-11-18 05:36:3= 5,681 INFO =C2=A0[AMRM Callback Handler Thread]
yarn.GiraphApplicationMa= ster
(GiraphApplicationMaster.java:onContainersAllocated(605)) - Got
= response from RM for container ask, allocatedCnt=3D3D1
2014-11-18 05:36:= 35,681 INFO =C2=A0[AMRM Callback Handler Thread]
yarn.GiraphApplicationM= aster
(GiraphApplicationMaster.java:onContainersAllocated(608)) - Total<= br>allocated # of container so far : 1 allocated out of 2 required.
2014= -11-18 05:36:35,681 INFO =C2=A0[AMRM Callback Handler Thread]
yarn.Girap= hApplicationMaster
(GiraphApplicationMaster.java:startContainerLaunching= Threads(359)) -
Launching command on a new container.,
containerId=3D= 3Dcontainer_1416310496070_0001_01_000002,
containerNode=3D3Droosevelt:60= 486, containerNodeURI=3D3Droosevelt:8042,
containerResourceMemory=3D3D10= 24
2014-11-18 05:36:35,688 INFO =C2=A0[pool-4-thread-1]
yarn.GiraphAp= plicationMaster
(GiraphApplicationMaster.java:buildContainerLaunchContex= t(492)) -
Setting up container launch container for
containerid=3D3Dc= ontainer_1416310496070_0001_01_000002
2014-11-18 05:36:35,720 INFO =C2= =A0[pool-4-thread-1]
yarn.GiraphApplicationMaster
(GiraphApplicationM= aster.java:buildContainerLaunchContext(498)) -
Conatain launch Commands = :java -Xmx1024M -Xms1024M -cp .:${CLASSPATH}
org.apache.giraph.yarn.Gira= phYarnTask 1416310496070 1 2 1
1><LOG_DIR>/task-2-stdout.log 2&= gt;<LOG_DIR>/task-2-stderr.log
2014-11-18 05:36:35,721 INFO =C2=A0= [pool-4-thread-1]
yarn.GiraphApplicationMaster
(GiraphApplicationMast= er.java:buildContainerLaunchContext(518)) -
Setting username in Containe= rLaunchContext to: hduser
2014-11-18 05:36:36,789 INFO =C2=A0[AMRM Callb= ack Handler Thread]
yarn.GiraphApplicationMaster
(GiraphApplicationMa= ster.java:onContainersAllocated(605)) - Got
response from RM for contain= er ask, allocatedCnt=3D3D1
2014-11-18 05:36:36,789 INFO =C2=A0[AMRM Call= back Handler Thread]
yarn.GiraphApplicationMaster
(GiraphApplicationM= aster.java:onContainersAllocated(608)) - Total
allocated # of container = so far : 2 allocated out of 2 required.
2014-11-18 05:36:36,790 INFO =C2= =A0[AMRM Callback Handler Thread]
yarn.GiraphApplicationMaster
(Girap= hApplicationMaster.java:startContainerLaunchingThreads(359)) -
Launching= command on a new container.,
containerId=3D3Dcontainer_1416310496070_00= 01_01_000003,
containerNode=3D3Droosevelt:60486, containerNodeURI=3D3Dro= osevelt:8042,
containerResourceMemory=3D3D1024
2014-11-18 05:36:37,22= 7 INFO =C2=A0[pool-4-thread-2]
yarn.GiraphApplicationMaster
(GiraphAp= plicationMaster.java:buildContainerLaunchContext(492)) -
Setting up cont= ainer launch container for
containerid=3D3Dcontainer_1416310496070_0001_= 01_000003
2014-11-18 05:36:37,227 INFO =C2=A0[pool-4-thread-2]
yarn.G= iraphApplicationMaster
(GiraphApplicationMaster.java:buildContainerLaunc= hContext(498)) -
Conatain launch Commands :java -Xmx1024M -Xms1024M -cp = .:${CLASSPATH}
org.apache.giraph.yarn.GiraphYarnTask 1416310496070 1 3 1=
1><LOG_DIR>/task-3-stdout.log 2><LOG_DIR>/task-3-stde= rr.log
2014-11-18 05:36:37,227 INFO =C2=A0[pool-4-thread-2]
yarn.Gira= phApplicationMaster
(GiraphApplicationMaster.java:buildContainerLaunchCo= ntext(518)) -
Setting username in ContainerLaunchContext to: hduser
2= 014-11-18 05:36:37,463 INFO =C2=A0[pool-4-thread-1] yarn.YarnUtils
(Yarn= Utils.java:addFsResourcesToMap(72)) - Adding
giraph-examples-1.1.0-SNAPS= HOT-for-hadoop-2.5.1-jar-with-dependencies.jar
to LocalResources for export.to
hdfs://loca= lhost:9000/user/hduser/giraph_yarn_jar_cache/application_1416310=3D
= 496070_0001/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-depend= =3D
encies.jar
2014-11-18 05:36:37,678 INFO =C2=A0[pool-4-thread-1] y= arn.YarnUtils
(YarnUtils.java:addFileToResourceMap(160)) - Registered fi= le in
LocalResources ::
hdfs://localhost:9000/user/hduser/giraph_y= arn_jar_cache/application_1416310=3D
496070_0001/giraph-examples-1.1= .0-SNAPSHOT-for-hadoop-2.5.1-jar-with-depend=3D
encies.jar
2014-11-18= 05:36:37,678 WARN =C2=A0[pool-4-thread-1] yarn.YarnUtils
(YarnUtils.jav= a:addFsResourcesToMap(81)) - Job jars (-yj option)
didn't include gi= raph-core.
2014-11-18 05:36:37,680 INFO =C2=A0[pool-4-thread-1] yarn.Yar= nUtils
(YarnUtils.java:addFileToResourceMap(160)) - Registered file inLocalResources ::
hdfs://localhost:9000/user/hduser/giraph_yarn_jar= _cache/application_1416310=3D
496070_0001/giraph-conf.xml
2014-11= -18 05:36:37,694 INFO
[org.apache.hadoop.yarn.client.api.async.impl.NMCl= ientAsyncImpl #0]
impl.NMClientAsyncImpl (NMClientAsyncImpl.java:run(531= )) - Processing
Event EventType: START_CONTAINER for Container
contai= ner_1416310496070_0001_01_000002
2014-11-18 05:36:37,699 INFO
[org.ap= ache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl #1]
impl.NMClie= ntAsyncImpl (NMClientAsyncImpl.java:run(531)) - Processing
Event EventTy= pe: START_CONTAINER for Container
container_1416310496070_0001_01_000003=
2014-11-18 05:36:37,698 INFO
[org.apache.hadoop.yarn.client.api.asyn= c.impl.NMClientAsyncImpl #0]
impl.ContainerManagementProtocolProxy
(C= ontainerManagementProtocolProxy.java:newProxy(212)) - Opening proxy
: ro= osevelt:60486
2014-11-18 05:36:37,761 INFO
[org.apache.hadoop.yarn.cl= ient.api.async.impl.NMClientAsyncImpl #2]
impl.NMClientAsyncImpl (NMClie= ntAsyncImpl.java:run(531)) - Processing
Event EventType: QUERY_CONTAINER= for Container
container_1416310496070_0001_01_000003
2014-11-18 05:3= 6:37,771 INFO
[org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyn= cImpl #3]
impl.NMClientAsyncImpl (NMClientAsyncImpl.java:run(531)) - Pro= cessing
Event EventType: QUERY_CONTAINER for Container
container_1416= 310496070_0001_01_000002



On Thu, Nov 20, 2014 at 1:32 PM, Al= essandro Negro <alenegro81@yahoo.it>
wrote:

Hi = Das,
I mean as user you run Hadoop yarn resource manager.

Alessan= dro

Il giorno 20/nov/2014, alle ore 19:22, D Adams <dadamszx@gmail.com> hascritto:

Alessandro,
=C2=A0=C2=A0=C2=A0=C2=A0I'm not sure wh= at you mean, should I create a new user on my system
named 'yarn'= ;? I'm new to both hadoop and giraph, so I'm not sure.

Tript= i,
=C2=A0=C2=A0=C2=A0=C2=A0I'll get those logs as soon as I can.
=
Thank you both,

V/r
Das
On Nov 20, 2014 3:13 AM, "Ale= ssandro Negro" <alenegro81@yahoo.it> wrote:

H= i Tripti,
I agree that a more detailed error log could be useful.
Thanks,
Alessandro




Il giorno 19/nov/2014, alle ore 1= 5:54, Tripti Singh <tripti@yahoo-inc.co=3D
m>
ha scritto:

--_000_D092AB9C1CB42triptiyahooinccom_
Content-Type: te= xt/plain; charset=3D3D"Windows-1252"
Content-Transfer-Encoding= : quoted-printable

Hi Alessandro,

I guess u r talking about t= he documentation on Giraph webpage which
mention=3D3D
s addition of this new option which is more or less= mandatory for yarn
base=3D3D
= d profiles.
When I first ran the Giraph on yarn without the proper =3D3D= 96yj option,
there =3D3D
was n= o way I could figure out from the Application/Container logs that
this=3D3D
was the issue.
I think, it= =3D3D92ll be a good idea to have this message in the logs for
easy d=3D3D
ebugging.

Thanks,
Tri= pti.


From: Alessandro Negro <alenegro81@yahoo.it<mailto:alenegro81@yahoo.it

Reply-To: "user@giraph.apache.org<mailto:user@giraph.apa= che.org>"
<user@gir=3D3D
aph.apache.o= rg<mailt= o:user@giraph.apache.org>>
Date: Tuesday, November 18, 2014 at= 4:31 PM
To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" <
<= /blockquote>user@giraph= .ap=3D3D
ache.org<mailto:user@giraph.apache.org>>
Subject: Re: He= lp with Giraph on Yarn

Hi Eli,
no I think that the help message i= s clear enough, generally when I rea=3D
d
=3D3D93=3D3= D
jar=3D3D94 I always mean absolute path for j= ar, but this is a my persona=3D
l=
misun=3D3D
derstanding.

Thanks,
Alessandro


I= l giorno 16/nov/2014, alle ore 21:39, Eli Reisman <
apache.mailbox@gma= il.co=3D3D
m<mailto:apache.mailbox@gmail.com>= > ha scritto:

Sounds like you got it figured out since last time = I checked this list=3D
,
sor=3D3D
ry it was a pain. Feel free to drop a JIRA ticket if we can make = help
messa=3D3D
ge etc. for -y= j option more clear, there's lots to improve there.

On Sat, Nov = 8, 2014 at 7:26 AM, Alessandro Negro <alenegro81@yahoo.it
<mailt=3D= 3D
o:alenegro81@yahoo.it>> wrote:
Hi Tripti,
finally I was= able to run the test with success. It was an issue of
perm= iss=3D3D
ion since I was running as ale not as= yarn.
Let me say that now I=3D3D92m able to run graph examples on Yarn = 2.5.1.
This is=3D3D
the final = result:

14/11/08 16:24:00 INFO yarn.GiraphYarnClient: Completed Gira= ph:
org.apache.=3D3D
giraph.ex= amples.SimpleShortestPathsComputation: SUCCEEDED, total
run= ning ti=3D3D
me: 0 minutes, 21 seconds.

Many thanks for your support,
Alessandro

Il giorno 06/nov/20= 14, alle ore 15:16, Tripti Singh <
tripti@yahoo-inc.com<m=3D3D
=
ailto:tripti@yahoo-inc.com>> ha scritto:

I don't kn= ow if u have access to this node. But if u do, u could check
if t=3D3D
he file is indeed there and u have= access to it.

Sent from my iPhone

On 06-Nov-2014, at 6:12 pm= , "Alessandro Negro" <alenegro81@yahoo.it
<mailto:=3D3D
alenegro81@yahoo.it>> wrote:

You are right it = works, but now I receive the following error:

SLF4J: Class path cont= ains multiple SLF4J bindings.
SLF4J: Found binding in
[j= ar:file:/private/tmp/hadoop-yarn/nm-local-dir/use=3D3D

rcache/ale/appcache/application_1415264041937_00= 09/filecache/10/giraph-e=3D
xam=3D3D
ples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.ja= r!/org/slf4=3D
j/i=3D3D
mpl/StaticLogger= Binder.class]
SLF4J: Found binding in
[jar:file:/opt/yar= n/hadoop-2.5.1/share/hadoop/commo=3D3D
n/lib/s= lf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class=3D
]
SLF4J: See=C2=A0http://www.slf4j.org/codes.html#multiple_bindings=C2=A0fo= r an
explana=3D3D
tion.
SLF= 4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2014-1= 1-06 13:15:37.120 java[10158:1803] Unable to load realm info fro=3D
m
SCD=3D3D
ynamicStore
Excep= tion in thread "pool-4-thread-1" java.lang.IllegalStateException:=
Coul=3D3D
d not configure the= containerlaunch context for GiraphYarnTasks.
at
org.apa= che.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(Giraph=3D
=3D3D
ApplicationMaster.java:391)
at
org.apache.giraph.yarn.GiraphApplicationMaster.access$500(Giraph= Applicat=3D
=3D3D
ionMaster.java:78)
= at
org.apache.giraph.yarn.GiraphApplicationMaster$LaunchCon= tainerRunnable.b=3D
=3D3D
uildContainer= LaunchContext(GiraphApplicationMaster.java:522)
at
org.a= pache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.r=3D
<= /blockquote>
=3D3D
un(GiraphApplicationMaster.java:479)at
java.util.concurrent.ThreadPoolExecutor.runWorker(Thre= adPoolExecutor.jav=3D
=3D3D
a:1145)
a= t
java.util.concurrent.ThreadPoolExecutor$Worker.run(Thread= PoolExecutor.ja=3D
=3D3D
va:615)
at= java.lang.Thread.run(Thread.java:744)
Caused by: java.io.FileNotFoundEx= ception: File does not exist:
hdfs://hadoo=3D3D
<= blockquote type=3D"cite">
p-master:9000/user/yarn/giraph_ya= rn_jar_cache/application_1415264041937_=3D
000= =3D3D

9/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.= 5.1-jar-with-dependencies.=3D
jar
at
=
org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(Distrib= utedFileSy=3D
=3D3D
stem.java:1072)
a= t
org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(Di= stributedFileSy=3D
=3D3D
stem.java:106= 4)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolv= e(FileSystemLinkResolv=3D
=3D3D
er.java:= 81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getF= ileStatus(DistributedFi=3D
=3D3D
leSyste= m.java:1064)
at
org.apache.giraph.yarn.YarnUtils.addFile= ToResourceMap(YarnUtils.java:153=3D
=3D3D
<= blockquote type=3D"cite">
)
at
org.apache.giraph.yarn.YarnUtils.addFsResourcesTo= Map(YarnUtils.java:77)
at
org.= apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(Giraph=3D
=
=3D3D
ApplicationMaster.java:387)
... 6 = more


That justify the other error I receive in the task:
Coul= d not find or load main class org.apache.giraph.yarn.GiraphYarnTas=3D
k

Thanks,

Il giorno 0= 6/nov/2014, alle ore 13:07, Tripti Singh <
tripti@yahoo-inc.com<m= =3D3D
ailto:tripti@yahoo-inc.com>> ha scritto:

Why r= u adding two jars? Example jar ideally contains core library so
every=3D3D
thing should be available wit= h just one example jar included

Sent from my iPhone

On 06-Nov= -2014, at 4:33 pm, "Alessandro Negro" <alenegro81@yahoo.it
&= lt;mailto:=3D3D
alenegro81@yahoo.it>> wrote:

Hi= ,
now it seems better, I need to add:


giraph-1.1= .0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,giraph-=3D
exa=3D3D
mples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-j= ar-with-dependencies.jar


Now it seems that after a lot of cycle = it fail with this error:

Could not find or load main class org.apach= e.giraph.yarn.GiraphYarnTas=3D
k<= br>

But in this case the error appear in task-3-stderr.log =C2=A0not = in
gam-stderr.l=3D3D
og where = there is the following error:

LF4J: Class path contains multiple SLF= 4J bindings.
SLF4J: Found binding in
[jar:file:/private/= tmp/hadoop-yarn/nm-local-dir/use=3D3D

rcache/ale/appcache/application_1415264041937_0006/filecache/12/gi= raph-1=3D
.1.=3D3D

0-SN= APSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/St=3D
=
ati=3D3D
cLoggerBinder.class]
SLF4J: Fou= nd binding in
[jar:file:/private/tmp/hadoop-yarn/nm-local-d= ir/use=3D3D

rcache/ale/appcac= he/application_1415264041937_0006/filecache/10/giraph-e=3D
=
xam=3D3D

ples-1.1.0-SNAPSHOT-for-hadoop= -2.5.1-jar-with-dependencies.jar!/org/slf4=3D
= j/i=3D3D
mpl/StaticLoggerBinder.class]
SLF4J: Found binding in
=
[jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/commo=3D3D
n/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/Stati= cLoggerBinder.class=3D
]
SL= F4J: See=C2=A0http://www.slf4j.org/codes.html#multiple_bi= ndings=C2=A0for an
<= /div>



--089e01419f4e4f89250508761721--