giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From D Adams <dadam...@gmail.com>
Subject Re: Help with Giraph on Yarn
Date Thu, 20 Nov 2014 18:22:05 GMT
Alessandro,
     I'm not sure what you mean, should I create a new user on my system
named 'yarn'? I'm new to both hadoop and giraph, so I'm not sure.

Tripti,
     I'll get those logs as soon as I can.

Thank you both,

V/r
Das
On Nov 20, 2014 3:13 AM, "Alessandro Negro" <alenegro81@yahoo.it> wrote:

> Hi Tripti,
> I agree that a more detailed error log could be useful.
>
> Thanks,
> Alessandro
>
>
>
>
> Il giorno 19/nov/2014, alle ore 15:54, Tripti Singh <tripti@yahoo-inc.com>
> ha scritto:
>
> > --_000_D092AB9C1CB42triptiyahooinccom_
> > Content-Type: text/plain; charset="Windows-1252"
> > Content-Transfer-Encoding: quoted-printable
> >
> > Hi Alessandro,
> >
> > I guess u r talking about the documentation on Giraph webpage which
> mention=
> > s addition of this new option which is more or less mandatory for yarn
> base=
> > d profiles.
> > When I first ran the Giraph on yarn without the proper =96yj option,
> there =
> > was no way I could figure out from the Application/Container logs that
> this=
> > was the issue.
> > I think, it=92ll be a good idea to have this message in the logs for
> easy d=
> > ebugging.
> >
> > Thanks,
> > Tripti.
> >
> >
> > From: Alessandro Negro <alenegro81@yahoo.it<mailto:alenegro81@yahoo.it>>
> > Reply-To: "user@giraph.apache.org<mailto:user@giraph.apache.org>"
> <user@gir=
> > aph.apache.org<mailto:user@giraph.apache.org>>
> > Date: Tuesday, November 18, 2014 at 4:31 PM
> > To: "user@giraph.apache.org<mailto:user@giraph.apache.org>"
> <user@giraph.ap=
> > ache.org<mailto:user@giraph.apache.org>>
> > Subject: Re: Help with Giraph on Yarn
> >
> > Hi Eli,
> > no I think that the help message is clear enough, generally when I read
> =93=
> > jar=94 I always mean absolute path for jar, but this is a my personal
> misun=
> > derstanding.
> >
> > Thanks,
> > Alessandro
> >
> >
> > Il giorno 16/nov/2014, alle ore 21:39, Eli Reisman <
> apache.mailbox@gmail.co=
> > m<mailto:apache.mailbox@gmail.com>> ha scritto:
> >
> > Sounds like you got it figured out since last time I checked this list,
> sor=
> > ry it was a pain. Feel free to drop a JIRA ticket if we can make help
> messa=
> > ge etc. for -yj option more clear, there's lots to improve there.
> >
> > On Sat, Nov 8, 2014 at 7:26 AM, Alessandro Negro <alenegro81@yahoo.it
> <mailt=
> > o:alenegro81@yahoo.it>> wrote:
> > Hi Tripti,
> > finally I was able to run the test with success. It was an issue of
> permiss=
> > ion since I was running as ale not as yarn.
> > Let me say that now I=92m able to run graph examples on Yarn 2.5.1. This
> is=
> > the final result:
> >
> > 14/11/08 16:24:00 INFO yarn.GiraphYarnClient: Completed Giraph:
> org.apache.=
> > giraph.examples.SimpleShortestPathsComputation: SUCCEEDED, total running
> ti=
> > me: 0 minutes, 21 seconds.
> >
> >
> > Many thanks for your support,
> > Alessandro
> >
> > Il giorno 06/nov/2014, alle ore 15:16, Tripti Singh <
> tripti@yahoo-inc.com<m=
> > ailto:tripti@yahoo-inc.com>> ha scritto:
> >
> > I don't know if u have access to this node. But if u do, u could check
> if t=
> > he file is indeed there and u have access to it.
> >
> > Sent from my iPhone
> >
> > On 06-Nov-2014, at 6:12 pm, "Alessandro Negro" <alenegro81@yahoo.it
> <mailto:=
> > alenegro81@yahoo.it>> wrote:
> >
> > You are right it works, but now I receive the following error:
> >
> > SLF4J: Class path contains multiple SLF4J bindings.
> > SLF4J: Found binding in
> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/use=
> >
> rcache/ale/appcache/application_1415264041937_0009/filecache/10/giraph-exam=
> >
> ples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/i=
> > mpl/StaticLoggerBinder.class]
> > SLF4J: Found binding in
> [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/commo=
> > n/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explana=
> > tion.
> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> > 2014-11-06 13:15:37.120 java[10158:1803] Unable to load realm info from
> SCD=
> > ynamicStore
> > Exception in thread "pool-4-thread-1" java.lang.IllegalStateException:
> Coul=
> > d not configure the containerlaunch context for GiraphYarnTasks.
> > at
> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(Giraph=
> > ApplicationMaster.java:391)
> > at
> org.apache.giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicat=
> > ionMaster.java:78)
> > at
> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.b=
> > uildContainerLaunchContext(GiraphApplicationMaster.java:522)
> > at
> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.r=
> > un(GiraphApplicationMaster.java:479)
> > at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav=
> > a:1145)
> > at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja=
> > va:615)
> > at java.lang.Thread.run(Thread.java:744)
> > Caused by: java.io.FileNotFoundException: File does not exist:
> hdfs://hadoo=
> >
> p-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_000=
> >
> 9/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
> > at
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSy=
> > stem.java:1072)
> > at
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSy=
> > stem.java:1064)
> > at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolv=
> > er.java:81)
> > at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFi=
> > leSystem.java:1064)
> > at
> org.apache.giraph.yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153=
> > )
> > at
> org.apache.giraph.yarn.YarnUtils.addFsResourcesToMap(YarnUtils.java:77)
> > at
> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(Giraph=
> > ApplicationMaster.java:387)
> > ... 6 more
> >
> >
> > That justify the other error I receive in the task:
> > Could not find or load main class org.apache.giraph.yarn.GiraphYarnTask
> >
> > Thanks,
> >
> > Il giorno 06/nov/2014, alle ore 13:07, Tripti Singh <
> tripti@yahoo-inc.com<m=
> > ailto:tripti@yahoo-inc.com>> ha scritto:
> >
> > Why r u adding two jars? Example jar ideally contains core library so
> every=
> > thing should be available with just one example jar included
> >
> > Sent from my iPhone
> >
> > On 06-Nov-2014, at 4:33 pm, "Alessandro Negro" <alenegro81@yahoo.it
> <mailto:=
> > alenegro81@yahoo.it>> wrote:
> >
> > Hi,
> > now it seems better, I need to add:
> >
> >
> giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,giraph-exa=
> > mples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
> >
> >
> > Now it seems that after a lot of cycle it fail with this error:
> >
> > Could not find or load main class org.apache.giraph.yarn.GiraphYarnTask
> >
> > But in this case the error appear in task-3-stderr.log  not in
> gam-stderr.l=
> > og where there is the following error:
> >
> > LF4J: Class path contains multiple SLF4J bindings.
> > SLF4J: Found binding in
> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/use=
> >
> rcache/ale/appcache/application_1415264041937_0006/filecache/12/giraph-1.1.=
> >
> 0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/Stati=
> > cLoggerBinder.class]
> > SLF4J: Found binding in
> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/use=
> >
> rcache/ale/appcache/application_1415264041937_0006/filecache/10/giraph-exam=
> >
> ples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/i=
> > mpl/StaticLoggerBinder.class]
> > SLF4J: Found binding in
> [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/commo=
> > n/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explana=
> > tion.
> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> > 2014-11-06 11:31:25.186 java[8162:1803] Unable to load realm info from
> SCDy=
> > namicStore
> > Exception in thread "pool-4-thread-1" java.lang.IllegalStateException:
> Coul=
> > d not configure the containerlaunch context for GiraphYarnTasks.
> > at
> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(Giraph=
> > ApplicationMaster.java:391)
> > at
> org.apache.giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicat=
> > ionMaster.java:78)
> > at
> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.b=
> > uildContainerLaunchContext(GiraphApplicationMaster.java:522)
> > at
> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.r=
> > un(GiraphApplicationMaster.java:479)
> > at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav=
> > a:1145)
> > at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja=
> > va:615)
> > at java.lang.Thread.run(Thread.java:744)
> > Caused by: java.io.FileNotFoundException: File does not exist:
> hdfs://hadoo=
> >
> p-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_000=
> > 6/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
> > at
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSy=
> > stem.java:1072)
> > at
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSy=
> > stem.java:1064)
> > at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolv=
> > er.java:81)
> > at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFi=
> > leSystem.java:1064)
> > at
> org.apache.giraph.yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153=
> > )
> > at
> org.apache.giraph.yarn.YarnUtils.addFsResourcesToMap(YarnUtils.java:77)
> > at
> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(Giraph=
> > ApplicationMaster.java:387)
> > ... 6 more
> >
> >
> > Il giorno 06/nov/2014, alle ore 11:32, Tripti Singh <
> tripti@yahoo-inc.com<m=
> > ailto:tripti@yahoo-inc.com>> ha scritto:
> >
> > Hi,
> > I remember seeing this thread, but since i was already deep into fixing
> thi=
> > s issue in production, I didn't really wait for the patch, not even the
> mun=
> > ge symbol as I was not even able to compile the yarn profile( sorry for
> bei=
> > ng so ruthless to giraph :-( )
> > I did later cross check with the patch and it was quite similar to my
> chang=
> > e but sure it wouldn't work for all profiles(in fact some profiles may
> even=
> > break due to Hadoop class incompatibility ) I can share those changes
> with=
> > u once I get back to my computer but I don't think that could have
> caused =
> > this problem.
> > Just try the small changed I mentioned in my previous reply and let me
> know=
> > if that helped.
> >
> > Thanks,
> > Tripti
> > Sent from my iPhone
> >
> > On 06-Nov-2014, at 3:52 pm, "Alessandro Negro" <alenegro81@yahoo.it
> <mailto:=
> > alenegro81@yahoo.it>> wrote:
> >
> > Hi Tripti,
> > let me add one info, As you can see from this thread of email:
> >
> >
> http://mail-archives.apache.org/mod_mbox/giraph-user/201410.mbox/%3CC3F75A9=
> > 8-BDE8-4205-971F-1F7241E6E83F@yahoo.it%3E
> >
> > I had some issue while compiling for Hadoop 2.5.1 could be that into the
> pr=
> > ocess I loose something?
> >
> > Thanks again,
> > Alessandro
> >
> >
> >
> > Il giorno 06/nov/2014, alle ore 10:47, Alessandro Negro
> <alenegro81@yahoo.i=
> > t<mailto:alenegro81@yahoo.it>> ha scritto:
> >
> > Hi Tripti,
> > yes I run in this way:
> >
> >
> > $HADOOP_INSTALL/bin/hadoop jar
> /Users/ale/extprj/giraph/giraph-examples/tar=
> >
> get/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.j=
> > ar org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimpleShortest=
> > PathsComputation -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubl=
> > eVertexInputFormat -vip /tiny_graph.txt -vof
> org.apache.giraph.io.formats.I=
> > dWithValueTextOutputFormat -op /shortestpaths -w 1 -yj
> /Users/ale/extprj/gi=
> >
> raph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dep=
> >
> endencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-exampl=
> > es-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
> >
> > And this is the complete trace:
> >
> > 2014-11-06 10:06:05.638 java[7778:1903] Unable to load realm info from
> SCDy=
> > namicStore
> > 14/11/06 10:06:05 WARN util.NativeCodeLoader: Unable to load
> native-hadoop =
> > library for your platform... using builtin-java classes where applicable
> > 14/11/06 10:06:06 INFO utils.ConfigurationUtils: No edge input format
> speci=
> > fied. Ensure your InputFormat does not require one.
> > 14/11/06 10:06:06 INFO utils.ConfigurationUtils: No edge output format
> spec=
> > ified. Ensure your OutputFormat does not require one.
> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Final output path is:
> hdfs://=
> > hadoop-master:9000/shortestpaths
> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Running Client
> > 14/11/06 10:06:06 INFO client.RMProxy: Connecting to ResourceManager at
> /0.=
> > 0.0.0:8032<http://0.0.0.0:8032/>
> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got node report from ASM
> for,=
> > nodeId=3D10.61.137.28:59677<http://10.61.137.28:59677/>, nodeAddress
> 10.61=
> > .137.28:8042<http://10.61.137.28:8042/>, nodeRackName /default-rack,
> nodeNu=
> > mContainers 0
> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Defaulting per-task heap
> size=
> > to 1024MB.
> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Obtained new Application
> ID: =
> > application_1415264041937_0005
> > 14/11/06 10:06:06 INFO Configuration.deprecation: mapred.job.id<
> http://mapr=
> > ed.job.id/> is deprecated. Instead, use mapreduce.job.id<
> http://mapreduce.j=
> > ob.id/>
> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Set the environment for
> the a=
> > pplication master
> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Environment for AM
> :{CLASSPAT=
> >
> H=3D${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/comm=
> >
> on/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/=
> >
> hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/s=
> >
> hare/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED=
> >
> _HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/l=
> > ib/*}
> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: buildLocalResourceMap ....
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Registered file in LocalResources
> ::=
> >
> hdfs://hadoop-master:9000/user/ale/giraph_yarn_jar_cache/application_14152=
> > 64041937_0005/giraph-conf.xml
> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: LIB JARS
> :/Users/ale/extprj/g=
> >
> iraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-de=
> >
> pendencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examp=
> > les-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name .
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking .
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name null
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking null
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> /opt/yarn/hadoop-2.5=
> > .1/etc/hadoop
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
> /opt/yarn/hadoop-=
> > 2.5.1/etc/hadoop
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> /opt/yarn/hadoop-2.5=
> > .1/share/hadoop/common/lib/*
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
> /opt/yarn/hadoop-=
> > 2.5.1/share/hadoop/common/lib
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> /opt/yarn/hadoop-2.5=
> > .1/share/hadoop/common/*
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
> /opt/yarn/hadoop-=
> > 2.5.1/share/hadoop/common
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> /opt/yarn/hadoop-2.5=
> > .1/share/hadoop/hdfs
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
> /opt/yarn/hadoop-=
> > 2.5.1/share/hadoop/hdfs
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> /opt/yarn/hadoop-2.5=
> > .1/share/hadoop/hdfs/lib/*
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
> /opt/yarn/hadoop-=
> > 2.5.1/share/hadoop/hdfs/lib
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> /opt/yarn/hadoop-2.5=
> > .1/share/hadoop/hdfs/*
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
> /opt/yarn/hadoop-=
> > 2.5.1/share/hadoop/hdfs
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> /opt/yarn/hadoop-2.5=
> > .1/share/hadoop/yarn/lib/*
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
> /opt/yarn/hadoop-=
> > 2.5.1/share/hadoop/yarn/lib
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> /opt/yarn/hadoop-2.5=
> > .1/share/hadoop/yarn/*
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
> /opt/yarn/hadoop-=
> > 2.5.1/share/hadoop/yarn
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> /opt/yarn/hadoop-2.5=
> > .1/share/hadoop/mapreduce/lib/*
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
> /opt/yarn/hadoop-=
> > 2.5.1/share/hadoop/mapreduce/lib
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> /opt/yarn/hadoop-2.5=
> > .1/share/hadoop/mapreduce/*
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
> /opt/yarn/hadoop-=
> > 2.5.1/share/hadoop/mapreduce
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> /opt/yarn/hadoop-2.5=
> > .1/share/myLib/*.jar
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
> /opt/yarn/hadoop-=
> > 2.5.1/share/myLib
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> /Users/ale/extprj/gi=
> > raph/myGiraph/*.jar
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
> /Users/ale/extprj=
> > /giraph/myGiraph
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> /Users/ale/extprj/gi=
> > raph/myGiraph/lib/*.jar
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
> /Users/ale/extprj=
> > /giraph/myGiraph/lib
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> /contrib/capacity-sc=
> > heduler/*.jar
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
> /contrib/capacity=
> > -scheduler
> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient:
> ApplicationSumbissionContext =
> > for GiraphApplicationMaster launch container is populated.
> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application to
> ASM
> > 14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application
> applicati=
> > on_1415264041937_0005
> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after
> submissio=
> > n :application_1415264041937_0005
> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: GiraphApplicationMaster
> conta=
> > iner request was submitted to ResourceManager for job: Giraph:
> org.apache.g=
> > iraph.examples.SimpleShortestPathsComputation
> > 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: Giraph:
> org.apache.giraph.exa=
> > mples.SimpleShortestPathsComputation, Elapsed: 0,83 secs
> > 14/11/06 10:06:07 INFO yarn.GiraphYarnClient:
> appattempt_1415264041937_0005=
> > _000001, State: ACCEPTED, Containers used: 1
> > 14/11/06 10:06:10 ERROR yarn.GiraphYarnClient: Giraph:
> org.apache.giraph.ex=
> > amples.SimpleShortestPathsComputation reports FAILED state, diagnostics
> sho=
> > w: Application application_1415264041937_0005 failed 2 times due to AM
> Cont=
> > ainer for appattempt_1415264041937_0005_000002 exited with  exitCode: 1
> due=
> > to: Exception from container-launch: ExitCodeException exitCode=3D1:
> > ExitCodeException exitCode=3D1:
> > at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
> > at org.apache.hadoop.util.Shell.run(Shell.java:455)
> > at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702=
> > )
> > at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launc=
> > hContainer(DefaultContainerExecutor.java:195)
> > at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Cont=
> > ainerLaunch.call(ContainerLaunch.java:300)
> > at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Cont=
> > ainerLaunch.call(ContainerLaunch.java:81)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> > at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav=
> > a:1145)
> > at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja=
> > va:615)
> > at java.lang.Thread.run(Thread.java:744)
> >
> >
> > Container exited with a non-zero exit code 1
> > .Failing this attempt.. Failing the application.
> > 14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Cleaning up HDFS
> distributed =
> > cache directory for Giraph job.
> > 14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Completed Giraph:
> org.apache.=
> > giraph.examples.SimpleShortestPathsComputation: FAILED, total running
> time:=
> > 0 minutes, 3 seconds.
> > ale:~/extprj/giraph/myGiraph />yarn logs -applicationId
> application_1415264=
> > 041937_0005
> >
> > As you can see there is this piece:
> >
> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: LIB JARS
> :/Users/ale/extprj/g=
> >
> iraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-de=
> >
> pendencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examp=
> > les-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
> >
> >
> > But I continue to receive the error:
> >
> > Thanks,
> > Alessandro
> >
> >
> > Il giorno 06/nov/2014, alle ore 10:25, Tripti Singh <
> tripti@yahoo-inc.com<m=
> > ailto:tripti@yahoo-inc.com>> ha scritto:
> >
> > For GiraphApplicationMaster not found, the simple thing that worked for
> me =
> > was adding a -yj parameter to my command.
> > (hadoop jar {jar-file-path} {path to giraph runner} {computation-class}
> {bu=
> > nch of parameters} -yj {jar-file-name})
> > If u look at job logs,
> > Do u see a log statement defining "LIB JARS"?
> >
> > Sent from my iPhone
> >
> > On 06-Nov-2014, at 2:45 pm, "Alessandro Negro" <alenegro81@yahoo.it
> <mailto:=
> > alenegro81@yahoo.it>> wrote:
> >
> > Hi,
> > yea my logs shows that error. I follow this example:
> >
> >
> http://mail-archives.apache.org/mod_mbox/giraph-user/201408.mbox/%3C53F4C68=
> > 9.5060101@web.de%3E
> >
> > without success.
> > Then I try to add all jars I can from giraph adding the following lines
> int=
> > o the hadoop-env.sh:
> >
> > export
> MYLIB=3D/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/g=
> > iraph/myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar
> > export HADOOP_CLASSPATH=3D$HADOOP_CLASSPATH:$MYLIB
> >
> >
> > when I run the application now I see this:
> >
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
> /opt/yarn/hadoop-=
> > 2.5.1/share/hadoop/mapreduce
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> /opt/yarn/hadoop-2.5=
> > .1/share/myLib/*.jar
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
> /opt/yarn/hadoop-=
> > 2.5.1/share/myLib
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> /Users/ale/extprj/gi=
> > raph/myGiraph/*.jar
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
> /Users/ale/extprj=
> > /giraph/myGiraph
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
> /Users/ale/extprj/gi=
> > raph/myGiraph/lib/*.jar
> > 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
> /Users/ale/extprj=
> > /giraph/myGiraph/lib
> >
> >
> > But again the same error. What I think is that there is no problem in
> the l=
> > ibrary but while is starting up.
> > I saw this piece in the logs:
> >
> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient:
> ApplicationSumbissionContext =
> > for GiraphApplicationMaster launch container is populated.
> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application to
> ASM
> > 14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application
> applicati=
> > on_1415264041937_0005
> > 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after
> submissio=
> > n :application_141526
>
>

Mime
View raw message