Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8D2BFCDD1 for ; Fri, 21 Nov 2014 11:05:56 +0000 (UTC) Received: (qmail 68019 invoked by uid 500); 21 Nov 2014 11:05:56 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 67965 invoked by uid 500); 21 Nov 2014 11:05:56 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 67955 invoked by uid 99); 21 Nov 2014 11:05:56 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 21 Nov 2014 11:05:56 +0000 X-ASF-Spam-Status: No, hits=3.1 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [212.82.96.236] (HELO nm19-vm3.bullet.mail.ir2.yahoo.com) (212.82.96.236) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 21 Nov 2014 11:05:26 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.it; s=s2048; t=1416567917; bh=RyAxmOzbKw5LE62JEKI7UWdFF+TjCf2tKvkBuSAamZ0=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject; b=pCoYk3um1AwVwcFWTaiCmqP4gPeEaJ3kvcyksfjrYnLry5IStV77Zr7VAlnPg1JADj1zYLJGfZvL0RlQk6LXrVkHHwlCE2ZfcKse+SKmQexab1n0SuQmfPvFdSTrDsR3eaBOJuzrqqq8jmREepA3FfY/f5arfAx8uW2q23SJUa9aPHCzrG0/pPmsqY6t7IaIrNAezsWG8sxCSloBXeJfsOa7BNBi+CoTYL8fGqECpqWqQSfvbhQOtE+oIxUmjF2EaBfhVvlRRyEYOYNl2TRpx224txW42a+A/FUjL5siKDH0qeYVIySoLzRY87id/IWA3LwDOdhe19Taf2yibJMn5w== DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s2048; d=yahoo.it; b=Pw3aYPFbrrhpOSjqyTZQUFML1gK7w/YIfmkUU5vD05xbisF3Wdj9g28/8PsvAFFrJV0s4r9v0EnX6Dke/3NQQ/6if+uMQn2Zga/Hlm7Nmq9jxKsvA/xtycoiwE021lABwBTCACbLHuaOkH2+uXVdazVpYYQ0AlgMOIyyX0pGqchkELdpIg6FWD3H3jllWFXqldSjz7mwbYsOzEexJaI/7fcjKmKedZf/wmXTRb+upgoBd0r1pcjdRx21AoXZEbh5bKiFMxBJYsS6qVQ3QDVM3yvJEuibXwiH5lpA2uLNKBwrrSHBoqqIbRkJoDaD50LY3VVHgNMRwqe4zvJQCThDNg==; Received: from [212.82.98.50] by nm19.bullet.mail.ir2.yahoo.com with NNFMP; 21 Nov 2014 11:05:17 -0000 Received: from [46.228.39.84] by tm3.bullet.mail.ir2.yahoo.com with NNFMP; 21 Nov 2014 11:05:17 -0000 Received: from [127.0.0.1] by smtp121.mail.ir2.yahoo.com with NNFMP; 21 Nov 2014 11:05:17 -0000 X-Yahoo-Newman-Id: 141512.52389.bm@smtp121.mail.ir2.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: _wFo4N4VM1lTKF8tKkE0QlJLa5zqQLSL_p5Y5DKypbR2iOo oWO_UWnomF71a.M.PS0jlYEvYKaFaAlnISpNtH46rAxJ9ZU_BqehOOcapVzI nswJwawRz0H6RMtX5eJl90OlE..Rfpg0c8.Du472PAPt8iapCj9wkBfMPpj3 7o7B5Azpvmb4i0jGeYVHO_ANyb9v8JWkc_I29HKTnrPEePLgPDYTFwZvj_yN OQBstXIOHDf8I_.VreOZmsnHzV2dhMyMOliSu3TXkBd7a6oMOoH1oLdTGodm Xkffgee7f8CiRFc.NvpiD3PqNzQFDb7D9WXcPM4Sfg197ZYInNK88lEZ_SMD BROcwxMMlHL7GcRv_bw_k00V89K34qZNRYshaKYffIBdX4KSzO9ShAbjUXs6 23aseGcKmnGe6Yer2eDvdRUDn9.DhrvQnNbuvSd47Db2e6epotWE3J0a.0p_ OLcsa6RqnLq3oMfl6l9BbWfhLSvodvlaeWGjy5rxSkIAjJYvquoQfMTwie7V cBEduQ3nRyYtnFpEXsR.FPs5sq1Zhz079DDrJZ5t0duOKbM3_yAcyCWo3iQI Fnfu_UMu2Kgc.lxtjSRu_HdoldOm8yMycjJvFuN.WUFMmfIZXufTnOSO5R6U 4VWjgJv_XDsN8WkXdmYvsNedq5C3Sp5UF3C_tOS9qid.l1EcRbFES6cutPC1 VABaRxxG8v5HvLap2yKY83FNbbfLISxZsPXJApEvffmdJ_VuOjTQjZgrSJdc YiQIlK5O7xRIz9usC91jZ6c1_4K7rneLJo2incqKV8nbsOsLbvwEYdDFt65R h3Dp8FPTuexkEXBUT5L.NGZuoRhSHWyjv3qY9sH9fy1LqR2zaJc_UbyQOcrA er5Zr5QvMOti_RMpm4ElpnfKruVI38xJURO7pWq7LG1AL_YLrCHtVEM8WIcF 7sGWQ9dObOtSO4pQ6bi4Na9cmDZHvZQklHA-- X-Yahoo-SMTP: qoe0kIaswBCBQnIixtWAKnaaI7JzLFw- From: Alessandro Negro Content-Type: multipart/alternative; boundary="Apple-Mail=_6EA07FC7-2342-4080-B7C8-04DB9B9252C6" Message-Id: Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Help with Giraph on Yarn Date: Fri, 21 Nov 2014 12:05:13 +0100 References: <77A73EB3-7967-4AB4-B5E6-191B46898119@yahoo.it> <0B32BE05-B6B6-4FEF-B2E8-8B99A45F4A83@yahoo.it> <41384542-C90C-4573-992E-CDD5E29B5803@yahoo-inc.com> <536AC63F-4C87-4071-A1D9-D867DCC6769B@yahoo.it> <92F1CFC1-0961-49C3-AE1F-FE970BFA4405@yahoo.it> <89D518FA-00B2-40C4-B391-BDCFB6CA2B50@yahoo.it> To: user@giraph.apache.org In-Reply-To: X-Mailer: Apple Mail (2.1878.6) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_6EA07FC7-2342-4080-B7C8-04DB9B9252C6 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 Hi, I forgot to say that I added also this line:=20 export HADOOP_CLASSPATH=3D$HADOOP_CLASSPATH:$MYLIB Il giorno 21/nov/2014, alle ore 10:34, Alessandro Negro = ha scritto: > Hi Das, > this is my configuration of hadoop: >=20 > yarn 1029 0,1 1,2 3499572 209560 s001 S 10:29am = 0:06.04 = /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/bin/java = -Dproc_resourcemanager -Xmx500m = -Dhadoop.log.dir=3D/opt/yarn/hadoop-2.5.1/logs = -Dyarn.log.dir=3D/opt/yarn/hadoop-2.5.1/logs = -Dhadoop.log.file=3Dyarn-yarn-resourcemanager-MacBook-Pro-di-Alessandro.lo= cal.log = -Dyarn.log.file=3Dyarn-yarn-resourcemanager-MacBook-Pro-di-Alessandro.loca= l.log -Dyarn.home.dir=3D -Dyarn.id.str=3Dyarn = -Dhadoop.root.logger=3DINFO,RFA -Dyarn.root.logger=3DINFO,RFA = -Djava.library.path=3D/opt/yarn/hadoop-2.5.1/lib/native = -Dyarn.policy.file=3Dhadoop-policy.xml = -Dhadoop.log.dir=3D/opt/yarn/hadoop-2.5.1/logs = -Dyarn.log.dir=3D/opt/yarn/hadoop-2.5.1/logs = -Dhadoop.log.file=3Dyarn-yarn-resourcemanager-MacBook-Pro-di-Alessandro.lo= cal.log = -Dyarn.log.file=3Dyarn-yarn-resourcemanager-MacBook-Pro-di-Alessandro.loca= l.log -Dyarn.home.dir=3D/opt/yarn/hadoop-2.5.1 = -Dhadoop.home.dir=3D/opt/yarn/hadoop-2.5.1 -Dhadoop.root.logger=3DINFO,RFA= -Dyarn.root.logger=3DINFO,RFA = -Djava.library.path=3D/opt/yarn/hadoop-2.5.1/lib/native -classpath = /opt/yarn/hadoop-2.5.1/etc/hadoop:/opt/yarn/hadoop-2.5.1/etc/hadoop:/opt/y= arn/hadoop-2.5.1/etc/hadoop:/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib= /*:/opt/yarn/hadoop-2.5.1/share/hadoop/common/*:/opt/yarn/hadoop-2.5.1/sha= re/hadoop/hdfs:/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/*:/opt/yarn/ha= doop-2.5.1/share/hadoop/hdfs/*:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/li= b/*:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*:/opt/yarn/hadoop-2.5.1/shar= e/hadoop/mapreduce/lib/*:/opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/*::= /opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/myGiraph= /*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar:/contrib/capacity-sched= uler/*.jar:/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/gira= ph/myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar:/contrib/cap= acity-scheduler/*.jar:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*:/opt/yarn= /hadoop-2.5.1/share/hadoop/yarn/lib/*:/opt/yarn/hadoop-2.5.1/etc/hadoop/rm= -config/log4j.properties = org.apache.hadoop.yarn.server.resourcemanager.ResourceManager > yarn 1073 0,1 1,0 3347132 174644 s001 S 10:29am = 0:04.77 = /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/bin/java = -Dproc_nodemanager -Xmx500m -Dhadoop.log.dir=3D/opt/yarn/hadoop-2.5.1/logs= -Dyarn.log.dir=3D/opt/yarn/hadoop-2.5.1/logs = -Dhadoop.log.file=3Dyarn-yarn-nodemanager-MacBook-Pro-di-Alessandro.local.= log = -Dyarn.log.file=3Dyarn-yarn-nodemanager-MacBook-Pro-di-Alessandro.local.lo= g -Dyarn.home.dir=3D -Dyarn.id.str=3Dyarn -Dhadoop.root.logger=3DINFO,RFA = -Dyarn.root.logger=3DINFO,RFA = -Djava.library.path=3D/opt/yarn/hadoop-2.5.1/lib/native = -Dyarn.policy.file=3Dhadoop-policy.xml -server = -Dhadoop.log.dir=3D/opt/yarn/hadoop-2.5.1/logs = -Dyarn.log.dir=3D/opt/yarn/hadoop-2.5.1/logs = -Dhadoop.log.file=3Dyarn-yarn-nodemanager-MacBook-Pro-di-Alessandro.local.= log = -Dyarn.log.file=3Dyarn-yarn-nodemanager-MacBook-Pro-di-Alessandro.local.lo= g -Dyarn.home.dir=3D/opt/yarn/hadoop-2.5.1 = -Dhadoop.home.dir=3D/opt/yarn/hadoop-2.5.1 -Dhadoop.root.logger=3DINFO,RFA= -Dyarn.root.logger=3DINFO,RFA = -Djava.library.path=3D/opt/yarn/hadoop-2.5.1/lib/native -classpath = /opt/yarn/hadoop-2.5.1/etc/hadoop:/opt/yarn/hadoop-2.5.1/etc/hadoop:/opt/y= arn/hadoop-2.5.1/etc/hadoop:/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib= /*:/opt/yarn/hadoop-2.5.1/share/hadoop/common/*:/opt/yarn/hadoop-2.5.1/sha= re/hadoop/hdfs:/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/*:/opt/yarn/ha= doop-2.5.1/share/hadoop/hdfs/*:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/li= b/*:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*:/opt/yarn/hadoop-2.5.1/shar= e/hadoop/mapreduce/lib/*:/opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/*::= /opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/myGiraph= /*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar:/contrib/capacity-sched= uler/*.jar:/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/gira= ph/myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar:/contrib/cap= acity-scheduler/*.jar:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*:/opt/yarn= /hadoop-2.5.1/share/hadoop/yarn/lib/*:/opt/yarn/hadoop-2.5.1/etc/hadoop/nm= -config/log4j.properties = org.apache.hadoop.yarn.server.nodemanager.NodeManager >=20 > So I run it as yarn user in this way: >=20 > sudo -u yarn =85. >=20 > Moreover I have just notice that I added the following lines into the = hadoop-env.sh >=20 > hadoop-env.sh:export = MYLIB=3D/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/= myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar >=20 > Accordingly to this other thread: >=20 > = http://mail-archives.apache.org/mod_mbox/giraph-user/201408.mbox/%3C53F4C6= 89.5060101@web.de%3E >=20 > Let me know, > Alessandro >=20 >=20 > Il giorno 20/nov/2014, alle ore 23:51, D Adams ha = scritto: >=20 >> --f46d044283ecf837440508522939 >> Content-Type: text/plain; charset=3DUTF-8 >> Content-Transfer-Encoding: quoted-printable >>=20 >> Ok, in the following, I use bin/yarn. Doesn't seem like much has = changed: >>=20 >> hduser@Roosevelt:/usr/local/hadoop$ ./hduser_jobs.sh >> Running hduser script ... >> bin/yarn jar >> = /usr/local/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-fo= r=3D >> -hadoop-2.5.1-jar-with-dependencies.jar >> org.apache.giraph.GiraphRunner >> org.apache.giraph.examples.SimpleShortestPathsComputation -vif >> = org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >> -vip /user/hduser/input/tiny_graph.txt -vof >> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >> /user/hduser/output/shortestpaths -w 1 -yj >> = giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >> 14/11/18 05:36:05 WARN util.NativeCodeLoader: Unable to load = native-hadoop >> library for your platform... using builtin-java classes where = applicable >> 14/11/18 05:36:08 INFO utils.ConfigurationUtils: No edge input format >> specified. Ensure your InputFormat does not require one. >> 14/11/18 05:36:08 INFO utils.ConfigurationUtils: No edge output = format >> specified. Ensure your OutputFormat does not require one. >> 14/11/18 05:36:08 INFO yarn.GiraphYarnClient: Final output path is: >> hdfs://localhost:9000/user/hduser/output/shortestpaths >> 14/11/18 05:36:08 INFO yarn.GiraphYarnClient: Running Client >> 14/11/18 05:36:08 INFO client.RMProxy: Connecting to ResourceManager = at >> localhost/127.0.0.1:8050 >> 14/11/18 05:36:09 INFO yarn.GiraphYarnClient: Got node report from = ASM for, >> nodeId=3D3Droosevelt:60486, nodeAddress roosevelt:8042, nodeRackName >> /default-rack, nodeNumContainers 0 >> 14/11/18 05:36:09 INFO yarn.GiraphYarnClient: Defaulting per-task = heap size >> to 1024MB. >> 14/11/18 05:36:09 INFO yarn.GiraphYarnClient: Obtained new = Application ID: >> application_1416310496070_0001 >> 14/11/18 05:36:09 INFO Configuration.deprecation: mapred.job.id is >> deprecated. Instead, use mapreduce.job.id >> 14/11/18 05:36:09 INFO yarn.GiraphYarnClient: Set the environment for = the >> application master >> 14/11/18 05:36:09 INFO yarn.GiraphYarnClient: Environment for AM >> = :{CLASSPATH=3D3D${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/shar= e/h=3D >> = adoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_= H=3D >> = OME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_= Y=3D >> = ARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HA= D=3D >> = OOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/= m=3D >> apreduce/lib/*} >> 14/11/18 05:36:09 INFO yarn.GiraphYarnClient: buildLocalResourceMap = .... >> 14/11/18 05:36:12 INFO yarn.YarnUtils: Registered file in = LocalResources :: >> = hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application_141631= 0=3D >> 496070_0001/giraph-conf.xml >> 14/11/18 05:36:12 INFO yarn.GiraphYarnClient: LIB JARS >> = :giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar= >> 14/11/18 05:36:12 INFO yarn.YarnUtils: Class path name . >> 14/11/18 05:36:12 INFO yarn.YarnUtils: base path checking . >> 14/11/18 05:36:12 INFO yarn.GiraphYarnClient: Made local resource for >> = :/usr/local/hadoop/share/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-j= a=3D >> r-with-dependencies.jar >> to >> = hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application_141631= 0=3D >> = 496070_0001/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-depen= d=3D >> encies.jar >> 14/11/18 05:36:16 INFO yarn.YarnUtils: Registered file in = LocalResources :: >> = hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application_141631= 0=3D >> = 496070_0001/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-depen= d=3D >> encies.jar >> 14/11/18 05:36:16 INFO yarn.GiraphYarnClient: = ApplicationSumbissionContext >> for GiraphApplicationMaster launch container is populated. >> 14/11/18 05:36:16 INFO yarn.GiraphYarnClient: Submitting application = to ASM >> 14/11/18 05:36:16 INFO impl.YarnClientImpl: Submitted application >> application_1416310496070_0001 >> 14/11/18 05:36:16 INFO yarn.GiraphYarnClient: Got new appId after >> submission :application_1416310496070_0001 >> 14/11/18 05:36:16 INFO yarn.GiraphYarnClient: GiraphApplicationMaster >> container request was submitted to ResourceManager for job: Giraph: >> org.apache.giraph.examples.SimpleShortestPathsComputation >> 14/11/18 05:36:17 INFO yarn.GiraphYarnClient: Giraph: >> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: = 1.01 >> secs >> 14/11/18 05:36:17 INFO yarn.GiraphYarnClient: >> appattempt_1416310496070_0001_000001, State: ACCEPTED, Containers = used: 1 >> 14/11/18 05:36:22 INFO yarn.GiraphYarnClient: Giraph: >> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: = 5.86 >> secs >> 14/11/18 05:36:22 INFO yarn.GiraphYarnClient: >> appattempt_1416310496070_0001_000001, State: ACCEPTED, Containers = used: 1 >> 14/11/18 05:36:26 INFO yarn.GiraphYarnClient: Giraph: >> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: = 10.02 >> secs >> 14/11/18 05:36:26 INFO yarn.GiraphYarnClient: >> appattempt_1416310496070_0001_000001, State: ACCEPTED, Containers = used: 1 >> 14/11/18 05:36:30 INFO yarn.GiraphYarnClient: Giraph: >> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: = 14.39 >> secs >> 14/11/18 05:36:30 INFO yarn.GiraphYarnClient: >> appattempt_1416310496070_0001_000001, State: ACCEPTED, Containers = used: 1 >> 14/11/18 05:36:34 INFO yarn.GiraphYarnClient: Giraph: >> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: = 18.57 >> secs >> 14/11/18 05:36:34 INFO yarn.GiraphYarnClient: >> appattempt_1416310496070_0001_000001, State: RUNNING, Containers = used: 2 >>=20 >> gam_stderr.log >>=20 >> LF4J: Class path contains multiple SLF4J bindings. >> SLF4J: Found binding in >> = [jar:file:/app/hadoop/tmp/nm-local-dir/usercache/hduser/appcache/applicati= o=3D >> = n_1416310496070_0001/filecache/10/giraph-examples-1.1.0-SNAPSHOT-for-hadoo= p=3D >> = -2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]= >> SLF4J: Found binding in >> = [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.ja= r=3D >> !/org/slf4j/impl/StaticLoggerBinder.class] >> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an = explana=3D >> tion. >> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] >>=20 >> gam_stdout.log >>=20 >> 2014-11-18 05:36:28,143 INFO [main] yarn.GiraphApplicationMaster >> (GiraphApplicationMaster.java:main(421)) - Starting GitaphAM >> 2014-11-18 05:36:30,404 WARN [main] util.NativeCodeLoader >> (NativeCodeLoader.java:(62)) - Unable to load native-hadoop >> library for your platform... using builtin-java classes where >> applicable >> 2014-11-18 05:36:32,649 INFO [main] yarn.GiraphApplicationMaster >> (GiraphApplicationMaster.java:(168)) - GiraphAM for = ContainerId >> container_1416310496070_0001_01_000001 ApplicationAttemptId >> appattempt_1416310496070_0001_000001 >> 2014-11-18 05:36:32,852 INFO [main] client.RMProxy >> (RMProxy.java:createRMProxy(92)) - Connecting to ResourceManager at >> localhost/127.0.0.1:8030 >> 2014-11-18 05:36:32,888 INFO [main] impl.NMClientAsyncImpl >> (NMClientAsyncImpl.java:serviceInit(107)) - Upper bound of the thread >> pool size is 500 >> 2014-11-18 05:36:32,889 INFO [main] >> impl.ContainerManagementProtocolProxy >> (ContainerManagementProtocolProxy.java:(78)) - >> yarn.client.max-nodemanagers-proxies : 500 >> 2014-11-18 05:36:33,589 INFO [main] yarn.GiraphApplicationMaster >> (GiraphApplicationMaster.java:setupContainerAskForRM(279)) - = Requested >> container ask: Capability[]Priority[10] >> 2014-11-18 05:36:33,617 INFO [main] yarn.GiraphApplicationMaster >> (GiraphApplicationMaster.java:setupContainerAskForRM(279)) - = Requested >> container ask: Capability[]Priority[10] >> 2014-11-18 05:36:33,617 INFO [main] yarn.GiraphApplicationMaster >> (GiraphApplicationMaster.java:run(185)) - Wait to finish .. >> 2014-11-18 05:36:35,674 INFO [AMRM Heartbeater thread] >> impl.AMRMClientImpl (AMRMClientImpl.java:populateNMTokens(299)) - >> Received new token for : roosevelt:60486 >> 2014-11-18 05:36:35,681 INFO [AMRM Callback Handler Thread] >> yarn.GiraphApplicationMaster >> (GiraphApplicationMaster.java:onContainersAllocated(605)) - Got >> response from RM for container ask, allocatedCnt=3D3D1 >> 2014-11-18 05:36:35,681 INFO [AMRM Callback Handler Thread] >> yarn.GiraphApplicationMaster >> (GiraphApplicationMaster.java:onContainersAllocated(608)) - Total >> allocated # of container so far : 1 allocated out of 2 required. >> 2014-11-18 05:36:35,681 INFO [AMRM Callback Handler Thread] >> yarn.GiraphApplicationMaster >> (GiraphApplicationMaster.java:startContainerLaunchingThreads(359)) - >> Launching command on a new container., >> containerId=3D3Dcontainer_1416310496070_0001_01_000002, >> containerNode=3D3Droosevelt:60486, containerNodeURI=3D3Droosevelt:8042,= >> containerResourceMemory=3D3D1024 >> 2014-11-18 05:36:35,688 INFO [pool-4-thread-1] >> yarn.GiraphApplicationMaster >> (GiraphApplicationMaster.java:buildContainerLaunchContext(492)) - >> Setting up container launch container for >> containerid=3D3Dcontainer_1416310496070_0001_01_000002 >> 2014-11-18 05:36:35,720 INFO [pool-4-thread-1] >> yarn.GiraphApplicationMaster >> (GiraphApplicationMaster.java:buildContainerLaunchContext(498)) - >> Conatain launch Commands :java -Xmx1024M -Xms1024M -cp .:${CLASSPATH} >> org.apache.giraph.yarn.GiraphYarnTask 1416310496070 1 2 1 >> 1>/task-2-stdout.log 2>/task-2-stderr.log >> 2014-11-18 05:36:35,721 INFO [pool-4-thread-1] >> yarn.GiraphApplicationMaster >> (GiraphApplicationMaster.java:buildContainerLaunchContext(518)) - >> Setting username in ContainerLaunchContext to: hduser >> 2014-11-18 05:36:36,789 INFO [AMRM Callback Handler Thread] >> yarn.GiraphApplicationMaster >> (GiraphApplicationMaster.java:onContainersAllocated(605)) - Got >> response from RM for container ask, allocatedCnt=3D3D1 >> 2014-11-18 05:36:36,789 INFO [AMRM Callback Handler Thread] >> yarn.GiraphApplicationMaster >> (GiraphApplicationMaster.java:onContainersAllocated(608)) - Total >> allocated # of container so far : 2 allocated out of 2 required. >> 2014-11-18 05:36:36,790 INFO [AMRM Callback Handler Thread] >> yarn.GiraphApplicationMaster >> (GiraphApplicationMaster.java:startContainerLaunchingThreads(359)) - >> Launching command on a new container., >> containerId=3D3Dcontainer_1416310496070_0001_01_000003, >> containerNode=3D3Droosevelt:60486, containerNodeURI=3D3Droosevelt:8042,= >> containerResourceMemory=3D3D1024 >> 2014-11-18 05:36:37,227 INFO [pool-4-thread-2] >> yarn.GiraphApplicationMaster >> (GiraphApplicationMaster.java:buildContainerLaunchContext(492)) - >> Setting up container launch container for >> containerid=3D3Dcontainer_1416310496070_0001_01_000003 >> 2014-11-18 05:36:37,227 INFO [pool-4-thread-2] >> yarn.GiraphApplicationMaster >> (GiraphApplicationMaster.java:buildContainerLaunchContext(498)) - >> Conatain launch Commands :java -Xmx1024M -Xms1024M -cp .:${CLASSPATH} >> org.apache.giraph.yarn.GiraphYarnTask 1416310496070 1 3 1 >> 1>/task-3-stdout.log 2>/task-3-stderr.log >> 2014-11-18 05:36:37,227 INFO [pool-4-thread-2] >> yarn.GiraphApplicationMaster >> (GiraphApplicationMaster.java:buildContainerLaunchContext(518)) - >> Setting username in ContainerLaunchContext to: hduser >> 2014-11-18 05:36:37,463 INFO [pool-4-thread-1] yarn.YarnUtils >> (YarnUtils.java:addFsResourcesToMap(72)) - Adding >> = giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >> to LocalResources for export.to >> = hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application_141631= 0=3D >> = 496070_0001/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-depen= d=3D >> encies.jar >> 2014-11-18 05:36:37,678 INFO [pool-4-thread-1] yarn.YarnUtils >> (YarnUtils.java:addFileToResourceMap(160)) - Registered file in >> LocalResources :: >> = hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application_141631= 0=3D >> = 496070_0001/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-depen= d=3D >> encies.jar >> 2014-11-18 05:36:37,678 WARN [pool-4-thread-1] yarn.YarnUtils >> (YarnUtils.java:addFsResourcesToMap(81)) - Job jars (-yj option) >> didn't include giraph-core. >> 2014-11-18 05:36:37,680 INFO [pool-4-thread-1] yarn.YarnUtils >> (YarnUtils.java:addFileToResourceMap(160)) - Registered file in >> LocalResources :: >> = hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/application_141631= 0=3D >> 496070_0001/giraph-conf.xml >> 2014-11-18 05:36:37,694 INFO >> [org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl #0] >> impl.NMClientAsyncImpl (NMClientAsyncImpl.java:run(531)) - Processing >> Event EventType: START_CONTAINER for Container >> container_1416310496070_0001_01_000002 >> 2014-11-18 05:36:37,699 INFO >> [org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl #1] >> impl.NMClientAsyncImpl (NMClientAsyncImpl.java:run(531)) - Processing >> Event EventType: START_CONTAINER for Container >> container_1416310496070_0001_01_000003 >> 2014-11-18 05:36:37,698 INFO >> [org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl #0] >> impl.ContainerManagementProtocolProxy >> (ContainerManagementProtocolProxy.java:newProxy(212)) - Opening proxy >> : roosevelt:60486 >> 2014-11-18 05:36:37,761 INFO >> [org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl #2] >> impl.NMClientAsyncImpl (NMClientAsyncImpl.java:run(531)) - Processing >> Event EventType: QUERY_CONTAINER for Container >> container_1416310496070_0001_01_000003 >> 2014-11-18 05:36:37,771 INFO >> [org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl #3] >> impl.NMClientAsyncImpl (NMClientAsyncImpl.java:run(531)) - Processing >> Event EventType: QUERY_CONTAINER for Container >> container_1416310496070_0001_01_000002 >>=20 >>=20 >>=20 >> On Thu, Nov 20, 2014 at 1:32 PM, Alessandro Negro = >> wrote: >>=20 >>> Hi Das, >>> I mean as user you run Hadoop yarn resource manager. >>>=20 >>> Alessandro >>>=20 >>> Il giorno 20/nov/2014, alle ore 19:22, D Adams = ha >>> scritto: >>>=20 >>> Alessandro, >>> I'm not sure what you mean, should I create a new user on my = system >>> named 'yarn'? I'm new to both hadoop and giraph, so I'm not sure. >>>=20 >>> Tripti, >>> I'll get those logs as soon as I can. >>>=20 >>> Thank you both, >>>=20 >>> V/r >>> Das >>> On Nov 20, 2014 3:13 AM, "Alessandro Negro" = wrote: >>>=20 >>>> Hi Tripti, >>>> I agree that a more detailed error log could be useful. >>>>=20 >>>> Thanks, >>>> Alessandro >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> Il giorno 19/nov/2014, alle ore 15:54, Tripti Singh = > m> >>>> ha scritto: >>>>=20 >>>>> --_000_D092AB9C1CB42triptiyahooinccom_ >>>>> Content-Type: text/plain; charset=3D3D"Windows-1252" >>>>> Content-Transfer-Encoding: quoted-printable >>>>>=20 >>>>> Hi Alessandro, >>>>>=20 >>>>> I guess u r talking about the documentation on Giraph webpage = which >>>> mention=3D3D >>>>> s addition of this new option which is more or less mandatory for = yarn >>>> base=3D3D >>>>> d profiles. >>>>> When I first ran the Giraph on yarn without the proper =3D3D96yj = option, >>>> there =3D3D >>>>> was no way I could figure out from the Application/Container logs = that >>>> this=3D3D >>>>> was the issue. >>>>> I think, it=3D3D92ll be a good idea to have this message in the = logs for >>>> easy d=3D3D >>>>> ebugging. >>>>>=20 >>>>> Thanks, >>>>> Tripti. >>>>>=20 >>>>>=20 >>>>> From: Alessandro Negro = >>>>>=20 >>>>> Reply-To: "user@giraph.apache.org" >>>> >>>> aph.apache.org> >>>>> Date: Tuesday, November 18, 2014 at 4:31 PM >>>>> To: "user@giraph.apache.org" < >>>> user@giraph.ap=3D3D >>>>> ache.org> >>>>> Subject: Re: Help with Giraph on Yarn >>>>>=20 >>>>> Hi Eli, >>>>> no I think that the help message is clear enough, generally when I = rea=3D >> d >>>> =3D3D93=3D3D >>>>> jar=3D3D94 I always mean absolute path for jar, but this is a my = persona=3D >> l >>>> misun=3D3D >>>>> derstanding. >>>>>=20 >>>>> Thanks, >>>>> Alessandro >>>>>=20 >>>>>=20 >>>>> Il giorno 16/nov/2014, alle ore 21:39, Eli Reisman < >>>> apache.mailbox@gmail.co=3D3D >>>>> m> ha scritto: >>>>>=20 >>>>> Sounds like you got it figured out since last time I checked this = list=3D >> , >>>> sor=3D3D >>>>> ry it was a pain. Feel free to drop a JIRA ticket if we can make = help >>>> messa=3D3D >>>>> ge etc. for -yj option more clear, there's lots to improve there. >>>>>=20 >>>>> On Sat, Nov 8, 2014 at 7:26 AM, Alessandro Negro = >>> >>>> o:alenegro81@yahoo.it>> wrote: >>>>> Hi Tripti, >>>>> finally I was able to run the test with success. It was an issue = of >>>> permiss=3D3D >>>>> ion since I was running as ale not as yarn. >>>>> Let me say that now I=3D3D92m able to run graph examples on Yarn = 2.5.1. >>>> This is=3D3D >>>>> the final result: >>>>>=20 >>>>> 14/11/08 16:24:00 INFO yarn.GiraphYarnClient: Completed Giraph: >>>> org.apache.=3D3D >>>>> giraph.examples.SimpleShortestPathsComputation: SUCCEEDED, total >>>> running ti=3D3D >>>>> me: 0 minutes, 21 seconds. >>>>>=20 >>>>>=20 >>>>> Many thanks for your support, >>>>> Alessandro >>>>>=20 >>>>> Il giorno 06/nov/2014, alle ore 15:16, Tripti Singh < >>>> tripti@yahoo-inc.com>>>> ailto:tripti@yahoo-inc.com>> ha scritto: >>>>>=20 >>>>> I don't know if u have access to this node. But if u do, u could = check >>>> if t=3D3D >>>>> he file is indeed there and u have access to it. >>>>>=20 >>>>> Sent from my iPhone >>>>>=20 >>>>> On 06-Nov-2014, at 6:12 pm, "Alessandro Negro" = >>> >>>> alenegro81@yahoo.it>> wrote: >>>>>=20 >>>>> You are right it works, but now I receive the following error: >>>>>=20 >>>>> SLF4J: Class path contains multiple SLF4J bindings. >>>>> SLF4J: Found binding in >>>> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/use=3D3D >>>>>=20 >>>> = rcache/ale/appcache/application_1415264041937_0009/filecache/10/giraph-e=3D= >> xam=3D3D >>>>>=20 >>>> = ples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4=3D= >> j/i=3D3D >>>>> mpl/StaticLoggerBinder.class] >>>>> SLF4J: Found binding in >>>> [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/commo=3D3D >>>>> = n/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class=3D >> ] >>>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for = an >>>> explana=3D3D >>>>> tion. >>>>> SLF4J: Actual binding is of type = [org.slf4j.impl.Log4jLoggerFactory] >>>>> 2014-11-06 13:15:37.120 java[10158:1803] Unable to load realm info = fro=3D >> m >>>> SCD=3D3D >>>>> ynamicStore >>>>> Exception in thread "pool-4-thread-1" = java.lang.IllegalStateException: >>>> Coul=3D3D >>>>> d not configure the containerlaunch context for GiraphYarnTasks. >>>>> at >>>> = org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(Giraph=3D= >> =3D3D >>>>> ApplicationMaster.java:391) >>>>> at >>>> = org.apache.giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicat=3D= >> =3D3D >>>>> ionMaster.java:78) >>>>> at >>>> = org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.b=3D= >> =3D3D >>>>> uildContainerLaunchContext(GiraphApplicationMaster.java:522) >>>>> at >>>> = org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.r=3D= >> =3D3D >>>>> un(GiraphApplicationMaster.java:479) >>>>> at >>>> = java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav=3D= >> =3D3D >>>>> a:1145) >>>>> at >>>> = java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja=3D= >> =3D3D >>>>> va:615) >>>>> at java.lang.Thread.run(Thread.java:744) >>>>> Caused by: java.io.FileNotFoundException: File does not exist: >>>> hdfs://hadoo=3D3D >>>>>=20 >>>> = p-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_=3D= >> 000=3D3D >>>>>=20 >>>> = 9/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.=3D= >> jar >>>>> at >>>> = org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSy=3D= >> =3D3D >>>>> stem.java:1072) >>>>> at >>>> = org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSy=3D= >> =3D3D >>>>> stem.java:1064) >>>>> at >>>> = org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolv=3D= >> =3D3D >>>>> er.java:81) >>>>> at >>>> = org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFi=3D= >> =3D3D >>>>> leSystem.java:1064) >>>>> at >>>> = org.apache.giraph.yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153=3D= >> =3D3D >>>>> ) >>>>> at >>>> = org.apache.giraph.yarn.YarnUtils.addFsResourcesToMap(YarnUtils.java:77) >>>>> at >>>> = org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(Giraph=3D= >> =3D3D >>>>> ApplicationMaster.java:387) >>>>> ... 6 more >>>>>=20 >>>>>=20 >>>>> That justify the other error I receive in the task: >>>>> Could not find or load main class = org.apache.giraph.yarn.GiraphYarnTas=3D >> k >>>>>=20 >>>>> Thanks, >>>>>=20 >>>>> Il giorno 06/nov/2014, alle ore 13:07, Tripti Singh < >>>> tripti@yahoo-inc.com>>>> ailto:tripti@yahoo-inc.com>> ha scritto: >>>>>=20 >>>>> Why r u adding two jars? Example jar ideally contains core library = so >>>> every=3D3D >>>>> thing should be available with just one example jar included >>>>>=20 >>>>> Sent from my iPhone >>>>>=20 >>>>> On 06-Nov-2014, at 4:33 pm, "Alessandro Negro" = >>> >>>> alenegro81@yahoo.it>> wrote: >>>>>=20 >>>>> Hi, >>>>> now it seems better, I need to add: >>>>>=20 >>>>>=20 >>>> = giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,giraph-=3D= >> exa=3D3D >>>>> mples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>=20 >>>>>=20 >>>>> Now it seems that after a lot of cycle it fail with this error: >>>>>=20 >>>>> Could not find or load main class = org.apache.giraph.yarn.GiraphYarnTas=3D >> k >>>>>=20 >>>>> But in this case the error appear in task-3-stderr.log not in >>>> gam-stderr.l=3D3D >>>>> og where there is the following error: >>>>>=20 >>>>> LF4J: Class path contains multiple SLF4J bindings. >>>>> SLF4J: Found binding in >>>> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/use=3D3D >>>>>=20 >>>> = rcache/ale/appcache/application_1415264041937_0006/filecache/12/giraph-1=3D= >> .1.=3D3D >>>>>=20 >>>> = 0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/St=3D= >> ati=3D3D >>>>> cLoggerBinder.class] >>>>> SLF4J: Found binding in >>>> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/use=3D3D >>>>>=20 >>>> = rcache/ale/appcache/application_1415264041937_0006/filecache/10/giraph-e=3D= >> xam=3D3D >>>>>=20 >>>> = ples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4=3D= >> j/i=3D3D >>>>> mpl/StaticLoggerBinder.class] >>>>> SLF4J: Found binding in >>>> [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/commo=3D3D >>>>> = n/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class=3D >> ] >>>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for = an >=20 --Apple-Mail=_6EA07FC7-2342-4080-B7C8-04DB9B9252C6 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252 Hi,
I forgot to say that I added also this = line: 

export = HADOOP_CLASSPATH=3D$HADOOP_CLASSPATH:$MYLIB


Il = giorno 21/nov/2014, alle ore 10:34, Alessandro Negro <alenegro81@yahoo.it> ha = scritto:

Hi = Das,
this is my configuration of = hadoop:

yarn           =   1029   0,1  1,2  3499572 209560 s001  S  =   10:29am   0:06.04 = /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/bin/java = -Dproc_resourcemanager -Xmx500m = -Dhadoop.log.dir=3D/opt/yarn/hadoop-2.5.1/logs = -Dyarn.log.dir=3D/opt/yarn/hadoop-2.5.1/logs = -Dhadoop.log.file=3Dyarn-yarn-resourcemanager-MacBook-Pro-di-Alessandro.lo= cal.log = -Dyarn.log.file=3Dyarn-yarn-resourcemanager-MacBook-Pro-di-Alessandro.loca= l.log -Dyarn.home.dir=3D -Dyarn.id.str=3Dyarn = -Dhadoop.root.logger=3DINFO,RFA -Dyarn.root.logger=3DINFO,RFA = -Djava.library.path=3D/opt/yarn/hadoop-2.5.1/lib/native = -Dyarn.policy.file=3Dhadoop-policy.xml = -Dhadoop.log.dir=3D/opt/yarn/hadoop-2.5.1/logs = -Dyarn.log.dir=3D/opt/yarn/hadoop-2.5.1/logs = -Dhadoop.log.file=3Dyarn-yarn-resourcemanager-MacBook-Pro-di-Alessandro.lo= cal.log = -Dyarn.log.file=3Dyarn-yarn-resourcemanager-MacBook-Pro-di-Alessandro.loca= l.log -Dyarn.home.dir=3D/opt/yarn/hadoop-2.5.1 = -Dhadoop.home.dir=3D/opt/yarn/hadoop-2.5.1 -Dhadoop.root.logger=3DINFO,RFA= -Dyarn.root.logger=3DINFO,RFA = -Djava.library.path=3D/opt/yarn/hadoop-2.5.1/lib/native -classpath = /opt/yarn/hadoop-2.5.1/etc/hadoop:/opt/yarn/hadoop-2.5.1/etc/hadoop:/opt/y= arn/hadoop-2.5.1/etc/hadoop:/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib= /*:/opt/yarn/hadoop-2.5.1/share/hadoop/common/*:/opt/yarn/hadoop-2.5.1/sha= re/hadoop/hdfs:/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/*:/opt/yarn/ha= doop-2.5.1/share/hadoop/hdfs/*:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/li= b/*:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*:/opt/yarn/hadoop-2.5.1/shar= e/hadoop/mapreduce/lib/*:/opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/*::= /opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/myGiraph= /*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar:/contrib/capacity-sched= uler/*.jar:/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/gira= ph/myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar:/contrib/cap= acity-scheduler/*.jar:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*:/opt/yarn= /hadoop-2.5.1/share/hadoop/yarn/lib/*:/opt/yarn/hadoop-2.5.1/etc/hadoop/rm= -config/log4j.properties = org.apache.hadoop.yarn.server.resourcemanager.ResourceManager
yarn   =           1073   0,1  1,0  = 3347132 174644 s001  S    10:29am   0:04.77 = /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/bin/java = -Dproc_nodemanager -Xmx500m -Dhadoop.log.dir=3D/opt/yarn/hadoop-2.5.1/logs= -Dyarn.log.dir=3D/opt/yarn/hadoop-2.5.1/logs = -Dhadoop.log.file=3Dyarn-yarn-nodemanager-MacBook-Pro-di-Alessandro.local.= log = -Dyarn.log.file=3Dyarn-yarn-nodemanager-MacBook-Pro-di-Alessandro.local.lo= g -Dyarn.home.dir=3D -Dyarn.id.str=3Dyarn -Dhadoop.root.logger=3DINFO,RFA = -Dyarn.root.logger=3DINFO,RFA = -Djava.library.path=3D/opt/yarn/hadoop-2.5.1/lib/native = -Dyarn.policy.file=3Dhadoop-policy.xml -server = -Dhadoop.log.dir=3D/opt/yarn/hadoop-2.5.1/logs = -Dyarn.log.dir=3D/opt/yarn/hadoop-2.5.1/logs = -Dhadoop.log.file=3Dyarn-yarn-nodemanager-MacBook-Pro-di-Alessandro.local.= log = -Dyarn.log.file=3Dyarn-yarn-nodemanager-MacBook-Pro-di-Alessandro.local.lo= g -Dyarn.home.dir=3D/opt/yarn/hadoop-2.5.1 = -Dhadoop.home.dir=3D/opt/yarn/hadoop-2.5.1 -Dhadoop.root.logger=3DINFO,RFA= -Dyarn.root.logger=3DINFO,RFA = -Djava.library.path=3D/opt/yarn/hadoop-2.5.1/lib/native -classpath = /opt/yarn/hadoop-2.5.1/etc/hadoop:/opt/yarn/hadoop-2.5.1/etc/hadoop:/opt/y= arn/hadoop-2.5.1/etc/hadoop:/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib= /*:/opt/yarn/hadoop-2.5.1/share/hadoop/common/*:/opt/yarn/hadoop-2.5.1/sha= re/hadoop/hdfs:/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/*:/opt/yarn/ha= doop-2.5.1/share/hadoop/hdfs/*:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/li= b/*:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*:/opt/yarn/hadoop-2.5.1/shar= e/hadoop/mapreduce/lib/*:/opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/*::= /opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/myGiraph= /*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar:/contrib/capacity-sched= uler/*.jar:/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/gira= ph/myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar:/contrib/cap= acity-scheduler/*.jar:/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*:/opt/yarn= /hadoop-2.5.1/share/hadoop/yarn/lib/*:/opt/yarn/hadoop-2.5.1/etc/hadoop/nm= -config/log4j.properties = org.apache.hadoop.yarn.server.nodemanager.NodeManager

=
So I run it as yarn user in this way:

sudo = -u yarn =85.

Moreover I have just notice that I = added the following lines into the = hadoop-env.sh

hadoop-env.sh:export = MYLIB=3D/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/= myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar

Accordingly to this other = thread:


Let me = know,
Alessandro


Il = giorno 20/nov/2014, alle ore 23:51, D Adams <dadamszx@gmail.com> ha = scritto:

--f46d044283ecf837440508522939
Content-Type: text/plain; = charset=3DUTF-8
Content-Transfer-Encoding: = quoted-printable

Ok, in the following, I use bin/yarn. Doesn't = seem like much has changed:

hduser@Roosevelt:/usr/local/hadoop$ = ./hduser_jobs.sh
Running hduser script ...
bin/yarn = jar
/usr/local/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAP= SHOT-for=3D
-hadoop-2.5.1-jar-with-dependencies.jar
org.apache.girap= h.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputatio= n = -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputF= ormat
-vip /user/hduser/input/tiny_graph.txt = -vof
org.apache.giraph.io.formats.IdWithValueTextOutputFormat = -op
/user/hduser/output/shortestpaths -w 1 = -yj
giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependenci= es.jar
14/11/18 05:36:05 WARN util.NativeCodeLoader: Unable to load = native-hadoop
library for your platform... using builtin-java classes = where applicable
14/11/18 05:36:08 INFO utils.ConfigurationUtils: No = edge input format
specified. Ensure your InputFormat does not require = one.
14/11/18 05:36:08 INFO utils.ConfigurationUtils: No edge output = format
specified. Ensure your OutputFormat does not require = one.
14/11/18 05:36:08 INFO yarn.GiraphYarnClient: Final output path = is:
hdfs://loc= alhost:9000/user/hduser/output/shortestpaths
14/11/18 05:36:08 = INFO yarn.GiraphYarnClient: Running Client
14/11/18 05:36:08 INFO = client.RMProxy: Connecting to ResourceManager = at
localhost/127.0.0.1:8050
14/11/18 05:36:09 INFO = yarn.GiraphYarnClient: Got node report from ASM = for,
nodeId=3D3Droosevelt:60486, nodeAddress roosevelt:8042, = nodeRackName
/default-rack, nodeNumContainers 0
14/11/18 05:36:09 = INFO yarn.GiraphYarnClient: Defaulting per-task heap size
to = 1024MB.
14/11/18 05:36:09 INFO yarn.GiraphYarnClient: Obtained new = Application ID:
application_1416310496070_0001
14/11/18 05:36:09 = INFO Configuration.deprecation: mapred.job.id is
deprecated. Instead, = use mapreduce.job.id
14/11/18 05:36:09 INFO yarn.GiraphYarnClient: = Set the environment for the
application master
14/11/18 05:36:09 = INFO yarn.GiraphYarnClient: Environment for = AM
:{CLASSPATH=3D3D${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOM= E/share/h=3D
adoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib= /*:$HADOOP_HDFS_H=3D
OME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/ha= doop/hdfs/lib/*:$HADOOP_Y=3D
ARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_= HOME/share/hadoop/yarn/lib/*:$HAD=3D
OOP_MAPRED_HOME/share/hadoop/mapre= duce/*:$HADOOP_MAPRED_HOME/share/hadoop/m=3D
apreduce/lib/*}
14/11/1= 8 05:36:09 INFO yarn.GiraphYarnClient: buildLocalResourceMap = ....
14/11/18 05:36:12 INFO yarn.YarnUtils: Registered file in = LocalResources ::
hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/appl= ication_1416310=3D
496070_0001/giraph-conf.xml
14/11/18 = 05:36:12 INFO yarn.GiraphYarnClient: LIB = JARS
:giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependen= cies.jar
14/11/18 05:36:12 INFO yarn.YarnUtils: Class path name = .
14/11/18 05:36:12 INFO yarn.YarnUtils: base path checking = .
14/11/18 05:36:12 INFO yarn.GiraphYarnClient: Made local resource = for
:/usr/local/hadoop/share/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-= 2.5.1-ja=3D
r-with-dependencies.jar
to
hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/appl= ication_1416310=3D
496070_0001/giraph-examples-1.1.0-SNAPSHOT-for-h= adoop-2.5.1-jar-with-depend=3D
encies.jar
14/11/18 05:36:16 INFO = yarn.YarnUtils: Registered file in LocalResources ::
hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/appl= ication_1416310=3D
496070_0001/giraph-examples-1.1.0-SNAPSHOT-for-h= adoop-2.5.1-jar-with-depend=3D
encies.jar
14/11/18 05:36:16 INFO = yarn.GiraphYarnClient: ApplicationSumbissionContext
for = GiraphApplicationMaster launch container is populated.
14/11/18 = 05:36:16 INFO yarn.GiraphYarnClient: Submitting application to = ASM
14/11/18 05:36:16 INFO impl.YarnClientImpl: Submitted = application
application_1416310496070_0001
14/11/18 05:36:16 INFO = yarn.GiraphYarnClient: Got new appId after
submission = :application_1416310496070_0001
14/11/18 05:36:16 INFO = yarn.GiraphYarnClient: GiraphApplicationMaster
container request was = submitted to ResourceManager for job: = Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation
14= /11/18 05:36:17 INFO yarn.GiraphYarnClient: = Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, = Elapsed: 1.01
secs
14/11/18 05:36:17 INFO = yarn.GiraphYarnClient:
appattempt_1416310496070_0001_000001, State: = ACCEPTED, Containers used: 1
14/11/18 05:36:22 INFO = yarn.GiraphYarnClient: = Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, = Elapsed: 5.86
secs
14/11/18 05:36:22 INFO = yarn.GiraphYarnClient:
appattempt_1416310496070_0001_000001, State: = ACCEPTED, Containers used: 1
14/11/18 05:36:26 INFO = yarn.GiraphYarnClient: = Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, = Elapsed: 10.02
secs
14/11/18 05:36:26 INFO = yarn.GiraphYarnClient:
appattempt_1416310496070_0001_000001, State: = ACCEPTED, Containers used: 1
14/11/18 05:36:30 INFO = yarn.GiraphYarnClient: = Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, = Elapsed: 14.39
secs
14/11/18 05:36:30 INFO = yarn.GiraphYarnClient:
appattempt_1416310496070_0001_000001, State: = ACCEPTED, Containers used: 1
14/11/18 05:36:34 INFO = yarn.GiraphYarnClient: = Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, = Elapsed: 18.57
secs
14/11/18 05:36:34 INFO = yarn.GiraphYarnClient:
appattempt_1416310496070_0001_000001, State: = RUNNING, Containers used: 2

gam_stderr.log

LF4J: Class = path contains multiple SLF4J bindings.
SLF4J: Found binding = in
[jar:file:/app/hadoop/tmp/nm-local-dir/usercache/hduser/appcache/app= licatio=3D
n_1416310496070_0001/filecache/10/giraph-examples-1.1.0-SNAP= SHOT-for-hadoop=3D
-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/Sta= ticLoggerBinder.class]
SLF4J: Found binding = in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.= 7.5.jar=3D
!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: = See http://www.slf4= j.org/codes.html#multiple_bindings for an = explana=3D
tion.
SLF4J: Actual binding is of type = [org.slf4j.impl.Log4jLoggerFactory]

gam_stdout.log

2014-11-1= 8 05:36:28,143 INFO  [main] = yarn.GiraphApplicationMaster
(GiraphApplicationMaster.java:main(421)) = - Starting GitaphAM
2014-11-18 05:36:30,404 WARN  [main] = util.NativeCodeLoader
(NativeCodeLoader.java:<clinit>(62)) - = Unable to load native-hadoop
library for your platform... using = builtin-java classes where
applicable
2014-11-18 05:36:32,649 INFO =  [main] = yarn.GiraphApplicationMaster
(GiraphApplicationMaster.java:<init>= (168)) - GiraphAM  for = ContainerId
container_1416310496070_0001_01_000001 = ApplicationAttemptId
appattempt_1416310496070_0001_000001
2014-11-18= 05:36:32,852 INFO  [main] = client.RMProxy
(RMProxy.java:createRMProxy(92)) - Connecting to = ResourceManager at
localhost/127.0.0.1:8030
2014-11-18 = 05:36:32,888 INFO  [main] = impl.NMClientAsyncImpl
(NMClientAsyncImpl.java:serviceInit(107)) - = Upper bound of the thread
pool size is 500
2014-11-18 05:36:32,889 = INFO =  [main]
impl.ContainerManagementProtocolProxy
(ContainerManagem= entProtocolProxy.java:<init>(78)) = -
yarn.client.max-nodemanagers-proxies : 500
2014-11-18 = 05:36:33,589 INFO  [main] = yarn.GiraphApplicationMaster
(GiraphApplicationMaster.java:setupContain= erAskForRM(279)) - Requested
container ask: = Capability[<memory:1024, vCores:0>]Priority[10]
2014-11-18 = 05:36:33,617 INFO  [main] = yarn.GiraphApplicationMaster
(GiraphApplicationMaster.java:setupContain= erAskForRM(279)) - Requested
container ask: = Capability[<memory:1024, vCores:0>]Priority[10]
2014-11-18 = 05:36:33,617 INFO  [main] = yarn.GiraphApplicationMaster
(GiraphApplicationMaster.java:run(185)) = - Wait to finish ..
2014-11-18 05:36:35,674 INFO  [AMRM = Heartbeater thread]
impl.AMRMClientImpl = (AMRMClientImpl.java:populateNMTokens(299)) -
Received new token for = : roosevelt:60486
2014-11-18 05:36:35,681 INFO  [AMRM Callback = Handler = Thread]
yarn.GiraphApplicationMaster
(GiraphApplicationMaster.java:o= nContainersAllocated(605)) - Got
response from RM for container ask, = allocatedCnt=3D3D1
2014-11-18 05:36:35,681 INFO  [AMRM Callback = Handler = Thread]
yarn.GiraphApplicationMaster
(GiraphApplicationMaster.java:o= nContainersAllocated(608)) - Total
allocated # of container so far : = 1 allocated out of 2 required.
2014-11-18 05:36:35,681 INFO =  [AMRM Callback Handler = Thread]
yarn.GiraphApplicationMaster
(GiraphApplicationMaster.java:s= tartContainerLaunchingThreads(359)) -
Launching command on a new = container.,
containerId=3D3Dcontainer_1416310496070_0001_01_000002,
= containerNode=3D3Droosevelt:60486, = containerNodeURI=3D3Droosevelt:8042,
containerResourceMemory=3D3D10242014-11-18 05:36:35,688 INFO =  [pool-4-thread-1]
yarn.GiraphApplicationMaster
(GiraphApplicat= ionMaster.java:buildContainerLaunchContext(492)) -
Setting up = container launch container = for
containerid=3D3Dcontainer_1416310496070_0001_01_000002
2014-11-1= 8 05:36:35,720 INFO =  [pool-4-thread-1]
yarn.GiraphApplicationMaster
(GiraphApplicat= ionMaster.java:buildContainerLaunchContext(498)) -
Conatain launch = Commands :java -Xmx1024M -Xms1024M -cp = .:${CLASSPATH}
org.apache.giraph.yarn.GiraphYarnTask 1416310496070 1 = 2 1
1><LOG_DIR>/task-2-stdout.log = 2><LOG_DIR>/task-2-stderr.log
2014-11-18 05:36:35,721 INFO =  [pool-4-thread-1]
yarn.GiraphApplicationMaster
(GiraphApplicat= ionMaster.java:buildContainerLaunchContext(518)) -
Setting username = in ContainerLaunchContext to: hduser
2014-11-18 05:36:36,789 INFO =  [AMRM Callback Handler = Thread]
yarn.GiraphApplicationMaster
(GiraphApplicationMaster.java:o= nContainersAllocated(605)) - Got
response from RM for container ask, = allocatedCnt=3D3D1
2014-11-18 05:36:36,789 INFO  [AMRM Callback = Handler = Thread]
yarn.GiraphApplicationMaster
(GiraphApplicationMaster.java:o= nContainersAllocated(608)) - Total
allocated # of container so far : = 2 allocated out of 2 required.
2014-11-18 05:36:36,790 INFO =  [AMRM Callback Handler = Thread]
yarn.GiraphApplicationMaster
(GiraphApplicationMaster.java:s= tartContainerLaunchingThreads(359)) -
Launching command on a new = container.,
containerId=3D3Dcontainer_1416310496070_0001_01_000003,
= containerNode=3D3Droosevelt:60486, = containerNodeURI=3D3Droosevelt:8042,
containerResourceMemory=3D3D10242014-11-18 05:36:37,227 INFO =  [pool-4-thread-2]
yarn.GiraphApplicationMaster
(GiraphApplicat= ionMaster.java:buildContainerLaunchContext(492)) -
Setting up = container launch container = for
containerid=3D3Dcontainer_1416310496070_0001_01_000003
2014-11-1= 8 05:36:37,227 INFO =  [pool-4-thread-2]
yarn.GiraphApplicationMaster
(GiraphApplicat= ionMaster.java:buildContainerLaunchContext(498)) -
Conatain launch = Commands :java -Xmx1024M -Xms1024M -cp = .:${CLASSPATH}
org.apache.giraph.yarn.GiraphYarnTask 1416310496070 1 = 3 1
1><LOG_DIR>/task-3-stdout.log = 2><LOG_DIR>/task-3-stderr.log
2014-11-18 05:36:37,227 INFO =  [pool-4-thread-2]
yarn.GiraphApplicationMaster
(GiraphApplicat= ionMaster.java:buildContainerLaunchContext(518)) -
Setting username = in ContainerLaunchContext to: hduser
2014-11-18 05:36:37,463 INFO =  [pool-4-thread-1] = yarn.YarnUtils
(YarnUtils.java:addFsResourcesToMap(72)) - = Adding
giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-depende= ncies.jar
to LocalResources for export.to
hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/appl= ication_1416310=3D
496070_0001/giraph-examples-1.1.0-SNAPSHOT-for-h= adoop-2.5.1-jar-with-depend=3D
encies.jar
2014-11-18 05:36:37,678 = INFO  [pool-4-thread-1] = yarn.YarnUtils
(YarnUtils.java:addFileToResourceMap(160)) - = Registered file in
LocalResources ::
hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/appl= ication_1416310=3D
496070_0001/giraph-examples-1.1.0-SNAPSHOT-for-h= adoop-2.5.1-jar-with-depend=3D
encies.jar
2014-11-18 05:36:37,678 = WARN  [pool-4-thread-1] = yarn.YarnUtils
(YarnUtils.java:addFsResourcesToMap(81)) - Job jars = (-yj option)
didn't include giraph-core.
2014-11-18 05:36:37,680 = INFO  [pool-4-thread-1] = yarn.YarnUtils
(YarnUtils.java:addFileToResourceMap(160)) - = Registered file in
LocalResources ::
hdfs://localhost:9000/user/hduser/giraph_yarn_jar_cache/appl= ication_1416310=3D
496070_0001/giraph-conf.xml
2014-11-18 = 05:36:37,694 = INFO
[org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl = #0]
impl.NMClientAsyncImpl (NMClientAsyncImpl.java:run(531)) - = Processing
Event EventType: START_CONTAINER for = Container
container_1416310496070_0001_01_000002
2014-11-18 = 05:36:37,699 = INFO
[org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl = #1]
impl.NMClientAsyncImpl (NMClientAsyncImpl.java:run(531)) - = Processing
Event EventType: START_CONTAINER for = Container
container_1416310496070_0001_01_000003
2014-11-18 = 05:36:37,698 = INFO
[org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl = #0]
impl.ContainerManagementProtocolProxy
(ContainerManagementProtoc= olProxy.java:newProxy(212)) - Opening proxy
: = roosevelt:60486
2014-11-18 05:36:37,761 = INFO
[org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl = #2]
impl.NMClientAsyncImpl (NMClientAsyncImpl.java:run(531)) - = Processing
Event EventType: QUERY_CONTAINER for = Container
container_1416310496070_0001_01_000003
2014-11-18 = 05:36:37,771 = INFO
[org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl = #3]
impl.NMClientAsyncImpl (NMClientAsyncImpl.java:run(531)) - = Processing
Event EventType: QUERY_CONTAINER for = Container
container_1416310496070_0001_01_000002



On = Thu, Nov 20, 2014 at 1:32 PM, Alessandro Negro <alenegro81@yahoo.it>
wrote:<= br>
Hi Das,
I mean as user you run = Hadoop yarn resource manager.

Alessandro

Il giorno = 20/nov/2014, alle ore 19:22, D Adams <dadamszx@gmail.com> = ha
scritto:

Alessandro,
    I'm not = sure what you mean, should I create a new user on my system
named = 'yarn'? I'm new to both hadoop and giraph, so I'm not = sure.

Tripti,
    I'll get those logs as = soon as I can.

Thank you both,

V/r
Das
On Nov 20, = 2014 3:13 AM, "Alessandro Negro" <alenegro81@yahoo.it> = wrote:

Hi Tripti,
I agree that a = more detailed error log could be = useful.

Thanks,
Alessandro




Il giorno = 19/nov/2014, alle ore 15:54, Tripti Singh <tripti@yahoo-inc.co=3D
m>
ha scritto:

--_000_D092AB9C1CB42triptiyahooinccom_
Content-Type: = text/plain; charset=3D3D"Windows-1252"
Content-Transfer-Encoding: = quoted-printable

Hi Alessandro,

I guess u r talking about = the documentation on Giraph webpage = which
mention=3D3D
s = addition of this new option which is more or less mandatory for = yarn
base=3D3D
d = profiles.
When I first ran the Giraph on yarn without the proper = =3D3D96yj option,
there =3D3D
was no way I could figure out from the = Application/Container logs that
this=3D3D
was the issue.
I think, it=3D3D92ll be a good idea to = have this message in the logs for
easy = d=3D3D
ebugging.

Thanks,
Tripti.


From: = Alessandro Negro <alenegro81@yahoo.it<mailto:alenegro81@yahoo.it

Reply-To: "user@giraph.apache.org<mailto:user@giraph.apache.org&g= t;"
<user@gir=3D3D
aph.apache.org<mailto:user@giraph.apache.org&g= t;>
Date: Tuesday, November 18, 2014 at 4:31 PM
To: "user@giraph.apache.org<mailto:user@giraph.apache.org&g= t;" <
user@giraph.ap=3D3D
ache.org<mailto:user@giraph.apache.org&g= t;>
Subject: Re: Help with Giraph on Yarn

Hi Eli,
no I = think that the help message is clear enough, generally when I = rea=3D
d
=3D3D93=3D3D
jar=3D3D94 I always mean absolute path for jar, but this = is a my = persona=3D
l
misun=3D3D
derstanding.

Thanks,
Alessandro


Il = giorno 16/nov/2014, alle ore 21:39, Eli Reisman <
apache.mailbox@gmail.co=3D3D
m<mailto:apache.mailbox@gmail.com>> ha scritto:

Sounds like you got it figured out since = last time I checked this = list=3D
,
sor=3D3D
ry it was a pain. Feel free to drop a JIRA ticket if we = can make help
messa=3D3D
ge = etc. for -yj option more clear, there's lots to improve there.

On = Sat, Nov 8, 2014 at 7:26 AM, Alessandro Negro <alenegro81@yahoo.it
<mailt=3D3D
o:alenegro81@yahoo.it>> wrote:
Hi = Tripti,
finally I was able to run the test with success. It was an = issue of
permiss=3D3D
ion = since I was running as ale not as yarn.
Let me say that now I=3D3D92m = able to run graph examples on Yarn 2.5.1.
This = is=3D3D
the final result:

14/11/08 = 16:24:00 INFO yarn.GiraphYarnClient: Completed = Giraph:
org.apache.=3D3D
giraph.examples.SimpleShortestPathsComputation: SUCCEEDED, = total
running ti=3D3D
me: 0 = minutes, 21 seconds.


Many thanks for your = support,
Alessandro

Il giorno 06/nov/2014, alle ore 15:16, = Tripti Singh <
tripti@yahoo-inc.com<m=3D3D
ailto:tripti@yahoo-inc.com>> ha = scritto:

I don't know if u have access to this node. But if u do, = u could check
if t=3D3D
he = file is indeed there and u have access to it.

Sent from my = iPhone

On 06-Nov-2014, at 6:12 pm, "Alessandro Negro" <alenegro81@yahoo.it
<mailto:=3D3D
alenegro81@yahoo.it>> = wrote:

You are right it works, but now I receive the following = error:

SLF4J: Class path contains multiple SLF4J = bindings.
SLF4J: Found binding = in
[jar:file:/private/tmp/hadoop-yarn/nm-local-dir/use=3D3= D

rcache/ale/appcache/application_14152640419= 37_0009/filecache/10/giraph-e=3D
xam=3D3D
=

ples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-wi= th-dependencies.jar!/org/slf4=3D
j/i=3D3D
=
mpl/StaticLoggerBinder.class]
SLF4J: Found binding = in
[jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/commo=3D3= D
n/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBi= nder.class=3D
]
SLF4J: = See http://www.slf4= j.org/codes.html#multiple_bindings for = an
explana=3D3D
tion.
SLF4J: Actual binding is of type = [org.slf4j.impl.Log4jLoggerFactory]
2014-11-06 13:15:37.120 = java[10158:1803] Unable to load realm info = fro=3D
m
SCD=3D3D
ynamicStore
Exception in thread "pool-4-thread-1" = java.lang.IllegalStateException:
Coul=3D3D
d not configure the containerlaunch context for = GiraphYarnTasks.
at
org.apache.giraph.yarn.GiraphApplic= ationMaster.getTaskResourceMap(Giraph=3D
=3D3= D
ApplicationMaster.java:391)
at
org.apache= .giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicat=3D
=3D3D
ionMaster.java:78)
at
org.apache.giraph.y= arn.GiraphApplicationMaster$LaunchContainerRunnable.b=3D
<= /blockquote>=3D3D
uildContainerLaunchContext(GiraphApplicationMaster.java:522)=
at
org.apache.giraph.yarn.GiraphApplicationMaster$Laun= chContainerRunnable.r=3D
=3D3D
un(GiraphApplicationMaster.java:479)
at
j= ava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav=3D=
=3D3D
a:1145)
at
java.util.concurrent.ThreadPoo= lExecutor$Worker.run(ThreadPoolExecutor.ja=3D
=3D3D
va:615)
at = java.lang.Thread.run(Thread.java:744)
Caused by: = java.io.FileNotFoundException: File does not exist:
hdfs://hadoo=3D3D

p-master:9000/user/yarn/giraph_yarn_jar_cac= he/application_1415264041937_=3D
000=3D3D
=

9/giraph-examples-1.1.0-SNAPSHOT-for-hadoop= -2.5.1-jar-with-dependencies.=3D
jar
at
org.apache.hadoop.hdfs.DistributedFileSys= tem$17.doCall(DistributedFileSy=3D
=3D3D
<= blockquote type=3D"cite">
stem.java:1072)
at
org.apache.hadoop.hdfs= .DistributedFileSystem$17.doCall(DistributedFileSy=3D
=3D3D
stem.java:1064)
at
org.apache.hadoop.fs.F= ileSystemLinkResolver.resolve(FileSystemLinkResolv=3D
=3D3D
er.java:81)
at
org.apache.hadoop.hdfs.Dis= tributedFileSystem.getFileStatus(DistributedFi=3D
=3D3D
leSystem.java:1064)
at
org.apache.giraph.= yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153=3D
=
=3D3D
)
at
org.apache.giraph.yarn.YarnUtils.add= FsResourcesToMap(YarnUtils.java:77)
at
org.apache.giraph.yarn.GiraphApplicationM= aster.getTaskResourceMap(Giraph=3D
=3D3D
<= blockquote type=3D"cite">
ApplicationMaster.java:387)
... 6 more


That = justify the other error I receive in the task:
Could not find or load = main class = org.apache.giraph.yarn.GiraphYarnTas=3D
k

Thanks,

Il giorno = 06/nov/2014, alle ore 13:07, Tripti Singh <
tripti@yahoo-inc.com<m=3D3D
ailto:tripti@yahoo-inc.com>> ha = scritto:

Why r u adding two jars? Example jar ideally contains = core library so
every=3D3D
thing should be available with just one example jar = included

Sent from my iPhone

On 06-Nov-2014, at 4:33 pm, = "Alessandro Negro" <alenegro81@yahoo.it
<mailto:=3D3D
alenegro81@yahoo.it>> = wrote:

Hi,
now it seems better, I need to = add:


giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-wi= th-dependencies.jar,giraph-=3D
exa=3D3D
mples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.= jar


Now it seems that after a lot of cycle it fail with this = error:

Could not find or load main class = org.apache.giraph.yarn.GiraphYarnTas=3D
k

But in this case the error = appear in task-3-stderr.log  not = in
gam-stderr.l=3D3D
og = where there is the following error:

LF4J: Class path contains = multiple SLF4J bindings.
SLF4J: Found binding = in
[jar:file:/private/tmp/hadoop-yarn/nm-local-dir/use=3D3= D

rcache/ale/appcache/application_14152640419= 37_0006/filecache/12/giraph-1=3D
.1.=3D3D
=

0-SNAPSHOT-for-hadoop-2.5.1-jar-with-depend= encies.jar!/org/slf4j/impl/St=3D
ati=3D3D
=
cLoggerBinder.class]
SLF4J: Found binding = in
[jar:file:/private/tmp/hadoop-yarn/nm-local-dir/use=3D3= D

rcache/ale/appcache/application_14152640419= 37_0006/filecache/10/giraph-e=3D
xam=3D3D
=

ples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-wi= th-dependencies.jar!/org/slf4=3D
j/i=3D3D
=
mpl/StaticLoggerBinder.class]
SLF4J: Found binding = in
[jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/commo=3D3= D
n/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBi= nder.class=3D
]
SLF4J: = See http://www.slf4= j.org/codes.html#multiple_bindings for = an


= --Apple-Mail=_6EA07FC7-2342-4080-B7C8-04DB9B9252C6--