giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alessandro Negro <alenegr...@yahoo.it>
Subject Re: Help with Giraph on Yarn
Date Thu, 20 Nov 2014 19:32:12 GMT
Hi Das,
I mean as user you run Hadoop yarn resource manager.

Alessandro

> Il giorno 20/nov/2014, alle ore 19:22, D Adams <dadamszx@gmail.com> ha scritto:
> 
> Alessandro,
>      I'm not sure what you mean, should I create a new user on my system named 'yarn'?
I'm new to both hadoop and giraph, so I'm not sure.
> 
> Tripti,
>      I'll get those logs as soon as I can.
> 
> Thank you both,
> 
> V/r
> Das
> 
>> On Nov 20, 2014 3:13 AM, "Alessandro Negro" <alenegro81@yahoo.it> wrote:
>> Hi Tripti,
>> I agree that a more detailed error log could be useful.
>> 
>> Thanks,
>> Alessandro
>> 
>> 
>> 
>> 
>> Il giorno 19/nov/2014, alle ore 15:54, Tripti Singh <tripti@yahoo-inc.com>
ha scritto:
>> 
>> > --_000_D092AB9C1CB42triptiyahooinccom_
>> > Content-Type: text/plain; charset="Windows-1252"
>> > Content-Transfer-Encoding: quoted-printable
>> >
>> > Hi Alessandro,
>> >
>> > I guess u r talking about the documentation on Giraph webpage which mention=
>> > s addition of this new option which is more or less mandatory for yarn base=
>> > d profiles.
>> > When I first ran the Giraph on yarn without the proper =96yj option, there =
>> > was no way I could figure out from the Application/Container logs that this=
>> > was the issue.
>> > I think, it=92ll be a good idea to have this message in the logs for easy d=
>> > ebugging.
>> >
>> > Thanks,
>> > Tripti.
>> >
>> >
>> > From: Alessandro Negro <alenegro81@yahoo.it<mailto:alenegro81@yahoo.it>>
>> > Reply-To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" <user@gir=
>> > aph.apache.org<mailto:user@giraph.apache.org>>
>> > Date: Tuesday, November 18, 2014 at 4:31 PM
>> > To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" <user@giraph.ap=
>> > ache.org<mailto:user@giraph.apache.org>>
>> > Subject: Re: Help with Giraph on Yarn
>> >
>> > Hi Eli,
>> > no I think that the help message is clear enough, generally when I read =93=
>> > jar=94 I always mean absolute path for jar, but this is a my personal misun=
>> > derstanding.
>> >
>> > Thanks,
>> > Alessandro
>> >
>> >
>> > Il giorno 16/nov/2014, alle ore 21:39, Eli Reisman <apache.mailbox@gmail.co=
>> > m<mailto:apache.mailbox@gmail.com>> ha scritto:
>> >
>> > Sounds like you got it figured out since last time I checked this list, sor=
>> > ry it was a pain. Feel free to drop a JIRA ticket if we can make help messa=
>> > ge etc. for -yj option more clear, there's lots to improve there.
>> >
>> > On Sat, Nov 8, 2014 at 7:26 AM, Alessandro Negro <alenegro81@yahoo.it<mailt=
>> > o:alenegro81@yahoo.it>> wrote:
>> > Hi Tripti,
>> > finally I was able to run the test with success. It was an issue of permiss=
>> > ion since I was running as ale not as yarn.
>> > Let me say that now I=92m able to run graph examples on Yarn 2.5.1. This is=
>> > the final result:
>> >
>> > 14/11/08 16:24:00 INFO yarn.GiraphYarnClient: Completed Giraph: org.apache.=
>> > giraph.examples.SimpleShortestPathsComputation: SUCCEEDED, total running ti=
>> > me: 0 minutes, 21 seconds.
>> >
>> >
>> > Many thanks for your support,
>> > Alessandro
>> >
>> > Il giorno 06/nov/2014, alle ore 15:16, Tripti Singh <tripti@yahoo-inc.com<m=
>> > ailto:tripti@yahoo-inc.com>> ha scritto:
>> >
>> > I don't know if u have access to this node. But if u do, u could check if t=
>> > he file is indeed there and u have access to it.
>> >
>> > Sent from my iPhone
>> >
>> > On 06-Nov-2014, at 6:12 pm, "Alessandro Negro" <alenegro81@yahoo.it<mailto:=
>> > alenegro81@yahoo.it>> wrote:
>> >
>> > You are right it works, but now I receive the following error:
>> >
>> > SLF4J: Class path contains multiple SLF4J bindings.
>> > SLF4J: Found binding in [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/use=
>> > rcache/ale/appcache/application_1415264041937_0009/filecache/10/giraph-exam=
>> > ples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/i=
>> > mpl/StaticLoggerBinder.class]
>> > SLF4J: Found binding in [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/commo=
>> > n/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explana=
>> > tion.
>> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>> > 2014-11-06 13:15:37.120 java[10158:1803] Unable to load realm info from SCD=
>> > ynamicStore
>> > Exception in thread "pool-4-thread-1" java.lang.IllegalStateException: Coul=
>> > d not configure the containerlaunch context for GiraphYarnTasks.
>> > at org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(Giraph=
>> > ApplicationMaster.java:391)
>> > at org.apache.giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicat=
>> > ionMaster.java:78)
>> > at org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.b=
>> > uildContainerLaunchContext(GiraphApplicationMaster.java:522)
>> > at org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.r=
>> > un(GiraphApplicationMaster.java:479)
>> > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav=
>> > a:1145)
>> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja=
>> > va:615)
>> > at java.lang.Thread.run(Thread.java:744)
>> > Caused by: java.io.FileNotFoundException: File does not exist: hdfs://hadoo=
>> > p-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_000=
>> > 9/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>> > at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSy=
>> > stem.java:1072)
>> > at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSy=
>> > stem.java:1064)
>> > at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolv=
>> > er.java:81)
>> > at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFi=
>> > leSystem.java:1064)
>> > at org.apache.giraph.yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153=
>> > )
>> > at org.apache.giraph.yarn.YarnUtils.addFsResourcesToMap(YarnUtils.java:77)
>> > at org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(Giraph=
>> > ApplicationMaster.java:387)
>> > ... 6 more
>> >
>> >
>> > That justify the other error I receive in the task:
>> > Could not find or load main class org.apache.giraph.yarn.GiraphYarnTask
>> >
>> > Thanks,
>> >
>> > Il giorno 06/nov/2014, alle ore 13:07, Tripti Singh <tripti@yahoo-inc.com<m=
>> > ailto:tripti@yahoo-inc.com>> ha scritto:
>> >
>> > Why r u adding two jars? Example jar ideally contains core library so every=
>> > thing should be available with just one example jar included
>> >
>> > Sent from my iPhone
>> >
>> > On 06-Nov-2014, at 4:33 pm, "Alessandro Negro" <alenegro81@yahoo.it<mailto:=
>> > alenegro81@yahoo.it>> wrote:
>> >
>> > Hi,
>> > now it seems better, I need to add:
>> >
>> > giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,giraph-exa=
>> > mples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>> >
>> >
>> > Now it seems that after a lot of cycle it fail with this error:
>> >
>> > Could not find or load main class org.apache.giraph.yarn.GiraphYarnTask
>> >
>> > But in this case the error appear in task-3-stderr.log  not in gam-stderr.l=
>> > og where there is the following error:
>> >
>> > LF4J: Class path contains multiple SLF4J bindings.
>> > SLF4J: Found binding in [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/use=
>> > rcache/ale/appcache/application_1415264041937_0006/filecache/12/giraph-1.1.=
>> > 0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/Stati=
>> > cLoggerBinder.class]
>> > SLF4J: Found binding in [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/use=
>> > rcache/ale/appcache/application_1415264041937_0006/filecache/10/giraph-exam=
>> > ples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/i=
>> > mpl/StaticLoggerBinder.class]
>> > SLF4J: Found binding in [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/commo=
>> > n/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explana=
>> > tion.
>> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>> > 2014-11-06 11:31:25.186 java[8162:1803] Unable to load realm info from SCDy=
>> > namicStore
>> > Exception in thread "pool-4-thread-1" java.lang.IllegalStateException: Coul=
>> > d not configure the containerlaunch context for GiraphYarnTasks.
>> > at org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(Giraph=
>> > ApplicationMaster.java:391)
>> > at org.apache.giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicat=
>> > ionMaster.java:78)
>> > at org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.b=
>> > uildContainerLaunchContext(GiraphApplicationMaster.java:522)
>> > at org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.r=
>> > un(GiraphApplicationMaster.java:479)
>> > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav=
>> > a:1145)
>> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja=
>> > va:615)
>> > at java.lang.Thread.run(Thread.java:744)
>> > Caused by: java.io.FileNotFoundException: File does not exist: hdfs://hadoo=
>> > p-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_000=
>> > 6/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>> > at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSy=
>> > stem.java:1072)
>> > at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSy=
>> > stem.java:1064)
>> > at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolv=
>> > er.java:81)
>> > at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFi=
>> > leSystem.java:1064)
>> > at org.apache.giraph.yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153=
>> > )
>> > at org.apache.giraph.yarn.YarnUtils.addFsResourcesToMap(YarnUtils.java:77)
>> > at org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(Giraph=
>> > ApplicationMaster.java:387)
>> > ... 6 more
>> >
>> >
>> > Il giorno 06/nov/2014, alle ore 11:32, Tripti Singh <tripti@yahoo-inc.com<m=
>> > ailto:tripti@yahoo-inc.com>> ha scritto:
>> >
>> > Hi,
>> > I remember seeing this thread, but since i was already deep into fixing thi=
>> > s issue in production, I didn't really wait for the patch, not even the mun=
>> > ge symbol as I was not even able to compile the yarn profile( sorry for bei=
>> > ng so ruthless to giraph :-( )
>> > I did later cross check with the patch and it was quite similar to my chang=
>> > e but sure it wouldn't work for all profiles(in fact some profiles may even=
>> > break due to Hadoop class incompatibility ) I can share those changes with=
>> > u once I get back to my computer but I don't think that could have caused =
>> > this problem.
>> > Just try the small changed I mentioned in my previous reply and let me know=
>> > if that helped.
>> >
>> > Thanks,
>> > Tripti
>> > Sent from my iPhone
>> >
>> > On 06-Nov-2014, at 3:52 pm, "Alessandro Negro" <alenegro81@yahoo.it<mailto:=
>> > alenegro81@yahoo.it>> wrote:
>> >
>> > Hi Tripti,
>> > let me add one info, As you can see from this thread of email:
>> >
>> > http://mail-archives.apache.org/mod_mbox/giraph-user/201410.mbox/%3CC3F75A9=
>> > 8-BDE8-4205-971F-1F7241E6E83F@yahoo.it%3E
>> >
>> > I had some issue while compiling for Hadoop 2.5.1 could be that into the pr=
>> > ocess I loose something?
>> >
>> > Thanks again,
>> > Alessandro
>> >
>> >
>> >
>> > Il giorno 06/nov/2014, alle ore 10:47, Alessandro Negro <alenegro81@yahoo.i=
>> > t<mailto:alenegro81@yahoo.it>> ha scritto:
>> >
>> > Hi Tripti,
>> > yes I run in this way:
>> >
>> >
>> > $HADOOP_INSTALL/bin/hadoop jar /Users/ale/extprj/giraph/giraph-examples/tar=
>> > get/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.j=
>> > ar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortest=
>> > PathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubl=
>> > eVertexInputFormat -vip /tiny_graph.txt -vof org.apache.giraph.io.formats.I=
>> > dWithValueTextOutputFormat -op /shortestpaths -w 1 -yj /Users/ale/extprj/gi=
>> > raph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dep=
>> > endencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-exampl=
>> > es-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>> >
>> > And this is the complete trace:
>> >
>> > 2014-11-06 10:06:05.638 java[7778:1903] Unable to load realm info from SCDy=
>> > namicStore
>> > 14/11/06 10:06:05 WARN util.NativeCodeLoader: Unable to load native-hadoop =
>> > library for your platform... using builtin-java classes where applicable
>> > 14/11/06 10:06:06 INFO utils.ConfigurationUtils: No edge input format speci=
>> > fied. Ensure your InputFormat does not require one.
>> > 14/11/06 10:06:06 INFO utils.ConfigurationUtils: No edge output format spec=
>> > ified. Ensure your OutputFormat does not require one.
>> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Final output path is: hdfs://=
>> > hadoop-master:9000/shortestpaths
>> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Running Client
>> > 14/11/06 10:06:06 INFO client.RMProxy: Connecting to ResourceManager at /0.=
>> > 0.0.0:8032<http://0.0.0.0:8032/>
>> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got node report from ASM for,=
>> > nodeId=3D10.61.137.28:59677<http://10.61.137.28:59677/>, nodeAddress 10.61=
>> > .137.28:8042<http://10.61.137.28:8042/>, nodeRackName /default-rack, nodeNu=
>> > mContainers 0
>> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Defaulting per-task heap size=
>> > to 1024MB.
>> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Obtained new Application ID: =
>> > application_1415264041937_0005
>> > 14/11/06 10:06:06 INFO Configuration.deprecation: mapred.job.id<http://mapr=
>> > ed.job.id/> is deprecated. Instead, use mapreduce.job.id<http://mapreduce.j=
>> > ob.id/>
>> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Set the environment for the a=
>> > pplication master
>> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Environment for AM :{CLASSPAT=
>> > H=3D${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/comm=
>> > on/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/=
>> > hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/s=
>> > hare/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED=
>> > _HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/l=
>> > ib/*}
>> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: buildLocalResourceMap ....
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Registered file in LocalResources ::=
>> > hdfs://hadoop-master:9000/user/ale/giraph_yarn_jar_cache/application_14152=
>> > 64041937_0005/giraph-conf.xml
>> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: LIB JARS :/Users/ale/extprj/g=
>> > iraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-de=
>> > pendencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examp=
>> > les-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name .
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking .
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name null
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking null
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5=
>> > .1/etc/hadoop
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-=
>> > 2.5.1/etc/hadoop
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5=
>> > .1/share/hadoop/common/lib/*
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-=
>> > 2.5.1/share/hadoop/common/lib
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5=
>> > .1/share/hadoop/common/*
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-=
>> > 2.5.1/share/hadoop/common
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5=
>> > .1/share/hadoop/hdfs
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-=
>> > 2.5.1/share/hadoop/hdfs
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5=
>> > .1/share/hadoop/hdfs/lib/*
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-=
>> > 2.5.1/share/hadoop/hdfs/lib
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5=
>> > .1/share/hadoop/hdfs/*
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-=
>> > 2.5.1/share/hadoop/hdfs
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5=
>> > .1/share/hadoop/yarn/lib/*
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-=
>> > 2.5.1/share/hadoop/yarn/lib
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5=
>> > .1/share/hadoop/yarn/*
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-=
>> > 2.5.1/share/hadoop/yarn
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5=
>> > .1/share/hadoop/mapreduce/lib/*
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-=
>> > 2.5.1/share/hadoop/mapreduce/lib
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5=
>> > .1/share/hadoop/mapreduce/*
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-=
>> > 2.5.1/share/hadoop/mapreduce
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5=
>> > .1/share/myLib/*.jar
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-=
>> > 2.5.1/share/myLib
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /Users/ale/extprj/gi=
>> > raph/myGiraph/*.jar
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /Users/ale/extprj=
>> > /giraph/myGiraph
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /Users/ale/extprj/gi=
>> > raph/myGiraph/lib/*.jar
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /Users/ale/extprj=
>> > /giraph/myGiraph/lib
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /contrib/capacity-sc=
>> > heduler/*.jar
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /contrib/capacity=
>> > -scheduler
>> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: ApplicationSumbissionContext =
>> > for GiraphApplicationMaster launch container is populated.
>> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application to ASM
>> > 14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application applicati=
>> > on_1415264041937_0005
>> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after submissio=
>> > n :application_1415264041937_0005
>> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: GiraphApplicationMaster conta=
>> > iner request was submitted to ResourceManager for job: Giraph: org.apache.g=
>> > iraph.examples.SimpleShortestPathsComputation
>> > 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: Giraph: org.apache.giraph.exa=
>> > mples.SimpleShortestPathsComputation, Elapsed: 0,83 secs
>> > 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: appattempt_1415264041937_0005=
>> > _000001, State: ACCEPTED, Containers used: 1
>> > 14/11/06 10:06:10 ERROR yarn.GiraphYarnClient: Giraph: org.apache.giraph.ex=
>> > amples.SimpleShortestPathsComputation reports FAILED state, diagnostics sho=
>> > w: Application application_1415264041937_0005 failed 2 times due to AM Cont=
>> > ainer for appattempt_1415264041937_0005_000002 exited with  exitCode: 1 due=
>> > to: Exception from container-launch: ExitCodeException exitCode=3D1:
>> > ExitCodeException exitCode=3D1:
>> > at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>> > at org.apache.hadoop.util.Shell.run(Shell.java:455)
>> > at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702=
>> > )
>> > at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launc=
>> > hContainer(DefaultContainerExecutor.java:195)
>> > at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Cont=
>> > ainerLaunch.call(ContainerLaunch.java:300)
>> > at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Cont=
>> > ainerLaunch.call(ContainerLaunch.java:81)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav=
>> > a:1145)
>> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja=
>> > va:615)
>> > at java.lang.Thread.run(Thread.java:744)
>> >
>> >
>> > Container exited with a non-zero exit code 1
>> > .Failing this attempt.. Failing the application.
>> > 14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Cleaning up HDFS distributed =
>> > cache directory for Giraph job.
>> > 14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Completed Giraph: org.apache.=
>> > giraph.examples.SimpleShortestPathsComputation: FAILED, total running time:=
>> > 0 minutes, 3 seconds.
>> > ale:~/extprj/giraph/myGiraph />yarn logs -applicationId application_1415264=
>> > 041937_0005
>> >
>> > As you can see there is this piece:
>> >
>> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: LIB JARS :/Users/ale/extprj/g=
>> > iraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-de=
>> > pendencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examp=
>> > les-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>> >
>> >
>> > But I continue to receive the error:
>> >
>> > Thanks,
>> > Alessandro
>> >
>> >
>> > Il giorno 06/nov/2014, alle ore 10:25, Tripti Singh <tripti@yahoo-inc.com<m=
>> > ailto:tripti@yahoo-inc.com>> ha scritto:
>> >
>> > For GiraphApplicationMaster not found, the simple thing that worked for me =
>> > was adding a -yj parameter to my command.
>> > (hadoop jar {jar-file-path} {path to giraph runner} {computation-class} {bu=
>> > nch of parameters} -yj {jar-file-name})
>> > If u look at job logs,
>> > Do u see a log statement defining "LIB JARS"?
>> >
>> > Sent from my iPhone
>> >
>> > On 06-Nov-2014, at 2:45 pm, "Alessandro Negro" <alenegro81@yahoo.it<mailto:=
>> > alenegro81@yahoo.it>> wrote:
>> >
>> > Hi,
>> > yea my logs shows that error. I follow this example:
>> >
>> > http://mail-archives.apache.org/mod_mbox/giraph-user/201408.mbox/%3C53F4C68=
>> > 9.5060101@web.de%3E
>> >
>> > without success.
>> > Then I try to add all jars I can from giraph adding the following lines int=
>> > o the hadoop-env.sh:
>> >
>> > export MYLIB=3D/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/g=
>> > iraph/myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar
>> > export HADOOP_CLASSPATH=3D$HADOOP_CLASSPATH:$MYLIB
>> >
>> >
>> > when I run the application now I see this:
>> >
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-=
>> > 2.5.1/share/hadoop/mapreduce
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /opt/yarn/hadoop-2.5=
>> > .1/share/myLib/*.jar
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /opt/yarn/hadoop-=
>> > 2.5.1/share/myLib
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /Users/ale/extprj/gi=
>> > raph/myGiraph/*.jar
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /Users/ale/extprj=
>> > /giraph/myGiraph
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name /Users/ale/extprj/gi=
>> > raph/myGiraph/lib/*.jar
>> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking /Users/ale/extprj=
>> > /giraph/myGiraph/lib
>> >
>> >
>> > But again the same error. What I think is that there is no problem in the l=
>> > ibrary but while is starting up.
>> > I saw this piece in the logs:
>> >
>> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: ApplicationSumbissionContext =
>> > for GiraphApplicationMaster launch container is populated.
>> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application to ASM
>> > 14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application applicati=
>> > on_1415264041937_0005
>> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after submissio=
>> > n :application_141526
>> 

Mime
View raw message