incubator-giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shaunak Kashyap <ycombina...@gmail.com>
Subject Re: Getting started
Date Fri, 03 Feb 2012 02:06:44 GMT
I'm sorry, but I don't follow and am a bit confused. Is there an example I
could look at?

What I'm trying to achieve is using the giraph-*.jar as a dependency of my
project. In my project I would like to define a class similar to
org.apache.giraph.examples.SimpleShortestPathsVertex. Then I would like to
create a jar from my project and use that with "hadoop jar ..." or
"bin/giraph ...".

Hope that made sense. Thank you,

Shaunak

On Thu, Feb 2, 2012 at 1:40 PM, David Garcia <dgarcia@potomacfusion.com>wrote:

> Before you run the jar, run mvn clean package from trunk.  After that, put
> the snapshot (or the with-dependencies) jar on the class path for your
> example.  It should work after that.
>
> From: Shaunak Kashyap <ycombinator@gmail.com>
> Reply-To: "giraph-user@incubator.apache.org" <
> giraph-user@incubator.apache.org>
> Date: Wed, 1 Feb 2012 20:44:09 -0600
> To: "giraph-user@incubator.apache.org" <giraph-user@incubator.apache.org>
> Subject: Re: Getting started
>
> Thanks, Jakob.
>
> I downloaded the source from
> https://svn.apache.org/repos/asf/incubator/giraph/trunk, built it using
> "mvn compile", then issued this command from the base directory (where
> pom.xml lives):
>
> $ ./bin/giraph ../giraph-example-shortest-path-with-libs.jar
> com.skyboximaging.janus.giraph.example.SkyboxShortestPathsVertex
>
> And I got back this error:
>
> Can't find Giraph jar.
>
> Digging into the contents of ./bin/giraph, it looks like its expecting the
> Giraph JAR in lib/ whereas "mvn compile" places it in target/.
>
> Perhaps I shouldn't be compiling from source but using the "regular Giraph
> .tar.gz" you mentioned in your email. I couldn't find it referenced from
> anywhere on http://incubator.apache.org/giraph/ (but maybe I haven't
> looked hard enough).
>
> Thanks,
>
> Shaunak
>
> On Wed, Feb 1, 2012 at 11:17 AM, Jakob Homan <jghoman@gmail.com> wrote:
>
>> We need to update the wiki; it's out of date.  Now we have bin/giraph
>> to run user jars.  Once you have the regular Giraph .tar.gz unpacked
>> on your machine and your vertex jar somewhere:
>> bin/giraph <your vertex jar> <your vertex class> <other params such
as
>> directories, formats, etc...>
>>
>>
>> On Wed, Feb 1, 2012 at 11:09 AM, Shaunak Kashyap <ycombinator@gmail.com>
>> wrote:
>> > Hi,
>> >
>> > I was able to run the shortest paths example as described
>> > here:
>> https://cwiki.apache.org/confluence/display/GIRAPH/Shortest+Paths+Example
>> .
>> >
>> > After this I tried to create my own JAR containing a class similar
>> > to SimpleShortestPathsVertex and a "lib" directory containing the
>> > giraph-*.jar (obtained from the target/ directory after building from
>> > source) and other dependent JARs. When I tried to run my JAR as a hadoop
>> > job, I get the following errors:
>> >
>> > $ hadoop jar giraph-example-shortest-path-with-libs.jar
>> > com.skyboximaging.janus.giraph.example.SkyboxShortestPathsVertex
>> > shortestPathsInputGraph shortestPathsOutputGraph 0 3
>> >
>> > 12/02/01 11:05:01 INFO mapred.JobClient: Running job:
>> job_201111161517_1471
>> > 12/02/01 11:05:02 INFO mapred.JobClient:  map 0% reduce 0%
>> > 12/02/01 11:05:07 INFO mapred.JobClient: Task Id :
>> > attempt_201111161517_1471_m_000001_0, Status : FAILED
>> > java.lang.IllegalStateException: run: Caught an unrecoverable exception
>> > java.lang.RuntimeException: java.lang.ClassNotFoundException:
>> > com.skyboximaging.janus.giraph.example.SkyboxShortestPathsVertex
>> >         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:650)
>> >         at
>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
>> >         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
>> >         at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
>> >         at java.security.AccessController.doPrivileged(Native Method)
>> >         at javax.security.auth.Subject.doAs(Subject.java:396)
>> >         at
>> >
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>> >         at org.apache.hadoop.mapred.Child.main(Child.java:264)
>> > Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
>> > java.lang.ClassNotFoundException:
>> > com.skyboximaging.janus.giraph.example.SkyboxShortestPathsVertex
>> >         at
>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1028)
>> >         at
>> > org.apache.giraph.graph.BspUtils.getVertexClass(BspUtils.java:299)
>> >
>> > When I checked, the class does exist in my JAR, in the right path:
>> >
>> > $ jar tvf giraph-example-shortest-path-with-libs.jar | grep
>> > SkyboxShortestPathsVertex
>> >   6029 Tue Jan 31 13:22:50 PST 2012
>> > com/skyboximaging/janus/giraph/example/SkyboxShortestPathsVertex.class
>> >
>> > Maybe this is yet another Hadoop issue (i.e. not specific to Giraph)
>> but I'd
>> > appreciate any help with it.
>> >
>> > Thank you,
>> >
>> > Shaunak
>> >
>> > On Wed, Dec 14, 2011 at 6:57 AM, Shaunak Kashyap <ycombinator@gmail.com
>> >
>> > wrote:
>> >>
>> >> Hi Avery,
>> >>
>> >> You're right; my error turned out to be a Hadoop issue which I have
>> >> worked around now. Thanks for the official documentation link.
>> >>
>> >> Shaunak
>> >>
>> >> On Tue, Dec 13, 2011 at 5:07 PM, Avery Ching <aching@apache.org>
>> wrote:
>> >> > Hi Shaunak,
>> >> >
>> >> > The official GIRAPH documentation is located at
>> >> > https://cwiki.apache.org/confluence/display/GIRAPH/Index .  That
>> being
>> >> > said,
>> >> > your error looks like a Hadoop issue.  Are you able to run normal
>> Hadoop
>> >> > example jobs?
>> >> >
>> >> > Thanks,
>> >> >
>> >> > Avery
>> >> >
>> >> >
>> >> > On 12/13/11 9:27 AM, Shaunak Kashyap wrote:
>> >> >>
>> >> >> Hi,
>> >> >>
>> >> >> Maybe this page -
>> >> >> https://github.com/aching/Giraph/wiki/Quick-Start-Guide - is
>> outdated
>> >> >> but I tried following the instructions there and am getting the
>> >> >> following error:
>> >> >>
>> >> >> shaunak@hadoop-1:~/Giraph$ hadoop jar
>> >> >> target/giraph-0.70-jar-with-dependencies.jar
>> >> >> org.apache.giraph.benchmark.PageRankBenchmark -h
>> >> >> 11/12/13 09:25:08 FATAL conf.Configuration: error parsing conf
file:
>> >> >> javax.xml.parsers.ParserConfigurationException: Feature
>> >> >> 'http://apache.org/xml/features/xinclude' is not recognized.
>> >> >> Exception in thread "main" java.lang.RuntimeException:
>> >> >> javax.xml.parsers.ParserConfigurationException: Feature
>> >> >> 'http://apache.org/xml/features/xinclude' is not recognized.
>> >> >>         at
>> >> >>
>> >> >>
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1396)
>> >> >>         at
>> >> >>
>> >> >>
>> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1251)
>> >> >>         at
>> >> >>
>> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1192)
>> >> >>         at
>> >> >> org.apache.hadoop.conf.Configuration.get(Configuration.java:415)
>> >> >>         at org.apache.hadoop.util.RunJar.main(RunJar.java:143)
>> >> >> Caused by: javax.xml.parsers.ParserConfigurationException: Feature
>> >> >> 'http://apache.org/xml/features/xinclude' is not recognized.
>> >> >>         at
>> >> >>
>> >> >>
>> org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown
>> >> >> Source)
>> >> >>         at
>> >> >>
>> >> >>
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1282)
>> >> >>         ... 4 more
>> >> >>
>> >> >> What am I missing?
>> >> >>
>> >> >> Thank you,
>> >> >>
>> >> >> Shaunak
>> >> >>
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> "Now the hardness of this world slowly grinds your dreams away /
>> >> Makin' a fool's joke out of the promises we make" --- Bruce
>> >> Springsteen, "Blood Brothers"
>> >
>> >
>> >
>> >
>> > --
>> > "Now the hardness of this world slowly grinds your dreams away / Makin'
>> a
>> > fool's joke out of the promises we make" --- Bruce Springsteen, "Blood
>> > Brothers"
>>
>
>
>
> --
> "Now the hardness of this world slowly grinds your dreams away / Makin' a
> fool's joke out of the promises we make" --- Bruce Springsteen, "Blood
> Brothers"
>



-- 
"Now the hardness of this world slowly grinds your dreams away / Makin' a
fool's joke out of the promises we make" --- Bruce Springsteen, "Blood
Brothers"

Mime
View raw message