incubator-giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jakob Homan <jgho...@gmail.com>
Subject Re: Getting started
Date Fri, 03 Feb 2012 20:15:24 GMT
Hey Shunak-
  Sorry, you've caught us with the place in a bit of a mess.  Until
GIRAPH-136 (which I just committed), it wasn't possible to run
bin/giraph from a fresh checkout of the source.  bin/giraph was
assumed to be run from the result of unpackaging of
giraph-<whatever>.tar.gz.  With GIRAPH-136, after running mvn:jar, it
should be possible to run bin/giraph (assuming a valid Hadoop
install).

Please let us know as you find these issues; we're working on
improving the user experience, but we're still at an early stage.
-Jakob


On Thu, Feb 2, 2012 at 6:06 PM, Shaunak Kashyap <ycombinator@gmail.com> wrote:
> I'm sorry, but I don't follow and am a bit confused. Is there an example I
> could look at?
>
> What I'm trying to achieve is using the giraph-*.jar as a dependency of my
> project. In my project I would like to define a class similar to
> org.apache.giraph.examples.SimpleShortestPathsVertex. Then I would like to
> create a jar from my project and use that with "hadoop jar ..." or
> "bin/giraph ...".
>
> Hope that made sense. Thank you,
>
> Shaunak
>
>
> On Thu, Feb 2, 2012 at 1:40 PM, David Garcia <dgarcia@potomacfusion.com>
> wrote:
>>
>> Before you run the jar, run mvn clean package from trunk.  After that, put
>> the snapshot (or the with-dependencies) jar on the class path for your
>> example.  It should work after that.
>>
>> From: Shaunak Kashyap <ycombinator@gmail.com>
>> Reply-To: "giraph-user@incubator.apache.org"
>> <giraph-user@incubator.apache.org>
>> Date: Wed, 1 Feb 2012 20:44:09 -0600
>> To: "giraph-user@incubator.apache.org" <giraph-user@incubator.apache.org>
>> Subject: Re: Getting started
>>
>> Thanks, Jakob.
>>
>> I downloaded the source
>> from https://svn.apache.org/repos/asf/incubator/giraph/trunk, built it using
>> "mvn compile", then issued this command from the base directory (where
>> pom.xml lives):
>>
>> $ ./bin/giraph ../giraph-example-shortest-path-with-libs.jar
>> com.skyboximaging.janus.giraph.example.SkyboxShortestPathsVertex
>>
>> And I got back this error:
>>
>> Can't find Giraph jar.
>>
>> Digging into the contents of ./bin/giraph, it looks like its expecting the
>> Giraph JAR in lib/ whereas "mvn compile" places it in target/.
>>
>> Perhaps I shouldn't be compiling from source but using the "regular Giraph
>> .tar.gz" you mentioned in your email. I couldn't find it referenced from
>> anywhere on http://incubator.apache.org/giraph/ (but maybe I haven't looked
>> hard enough).
>>
>> Thanks,
>>
>> Shaunak
>>
>> On Wed, Feb 1, 2012 at 11:17 AM, Jakob Homan <jghoman@gmail.com> wrote:
>>>
>>> We need to update the wiki; it's out of date.  Now we have bin/giraph
>>> to run user jars.  Once you have the regular Giraph .tar.gz unpacked
>>> on your machine and your vertex jar somewhere:
>>> bin/giraph <your vertex jar> <your vertex class> <other params
such as
>>> directories, formats, etc...>
>>>
>>>
>>> On Wed, Feb 1, 2012 at 11:09 AM, Shaunak Kashyap <ycombinator@gmail.com>
>>> wrote:
>>> > Hi,
>>> >
>>> > I was able to run the shortest paths example as described
>>> >
>>> > here: https://cwiki.apache.org/confluence/display/GIRAPH/Shortest+Paths+Example.
>>> >
>>> > After this I tried to create my own JAR containing a class similar
>>> > to SimpleShortestPathsVertex and a "lib" directory containing the
>>> > giraph-*.jar (obtained from the target/ directory after building from
>>> > source) and other dependent JARs. When I tried to run my JAR as a
>>> > hadoop
>>> > job, I get the following errors:
>>> >
>>> > $ hadoop jar giraph-example-shortest-path-with-libs.jar
>>> > com.skyboximaging.janus.giraph.example.SkyboxShortestPathsVertex
>>> > shortestPathsInputGraph shortestPathsOutputGraph 0 3
>>> >
>>> > 12/02/01 11:05:01 INFO mapred.JobClient: Running job:
>>> > job_201111161517_1471
>>> > 12/02/01 11:05:02 INFO mapred.JobClient:  map 0% reduce 0%
>>> > 12/02/01 11:05:07 INFO mapred.JobClient: Task Id :
>>> > attempt_201111161517_1471_m_000001_0, Status : FAILED
>>> > java.lang.IllegalStateException: run: Caught an unrecoverable exception
>>> > java.lang.RuntimeException: java.lang.ClassNotFoundException:
>>> > com.skyboximaging.janus.giraph.example.SkyboxShortestPathsVertex
>>> >         at
>>> > org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:650)
>>> >         at
>>> > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
>>> >         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
>>> >         at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
>>> >         at java.security.AccessController.doPrivileged(Native Method)
>>> >         at javax.security.auth.Subject.doAs(Subject.java:396)
>>> >         at
>>> >
>>> > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
>>> >         at org.apache.hadoop.mapred.Child.main(Child.java:264)
>>> > Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
>>> > java.lang.ClassNotFoundException:
>>> > com.skyboximaging.janus.giraph.example.SkyboxShortestPathsVertex
>>> >         at
>>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1028)
>>> >         at
>>> > org.apache.giraph.graph.BspUtils.getVertexClass(BspUtils.java:299)
>>> >
>>> > When I checked, the class does exist in my JAR, in the right path:
>>> >
>>> > $ jar tvf giraph-example-shortest-path-with-libs.jar | grep
>>> > SkyboxShortestPathsVertex
>>> >   6029 Tue Jan 31 13:22:50 PST 2012
>>> > com/skyboximaging/janus/giraph/example/SkyboxShortestPathsVertex.class
>>> >
>>> > Maybe this is yet another Hadoop issue (i.e. not specific to Giraph)
>>> > but I'd
>>> > appreciate any help with it.
>>> >
>>> > Thank you,
>>> >
>>> > Shaunak
>>> >
>>> > On Wed, Dec 14, 2011 at 6:57 AM, Shaunak Kashyap
>>> > <ycombinator@gmail.com>
>>> > wrote:
>>> >>
>>> >> Hi Avery,
>>> >>
>>> >> You're right; my error turned out to be a Hadoop issue which I have
>>> >> worked around now. Thanks for the official documentation link.
>>> >>
>>> >> Shaunak
>>> >>
>>> >> On Tue, Dec 13, 2011 at 5:07 PM, Avery Ching <aching@apache.org>
>>> >> wrote:
>>> >> > Hi Shaunak,
>>> >> >
>>> >> > The official GIRAPH documentation is located at
>>> >> > https://cwiki.apache.org/confluence/display/GIRAPH/Index .  That
>>> >> > being
>>> >> > said,
>>> >> > your error looks like a Hadoop issue.  Are you able to run normal
>>> >> > Hadoop
>>> >> > example jobs?
>>> >> >
>>> >> > Thanks,
>>> >> >
>>> >> > Avery
>>> >> >
>>> >> >
>>> >> > On 12/13/11 9:27 AM, Shaunak Kashyap wrote:
>>> >> >>
>>> >> >> Hi,
>>> >> >>
>>> >> >> Maybe this page -
>>> >> >> https://github.com/aching/Giraph/wiki/Quick-Start-Guide - is
>>> >> >> outdated
>>> >> >> but I tried following the instructions there and am getting
the
>>> >> >> following error:
>>> >> >>
>>> >> >> shaunak@hadoop-1:~/Giraph$ hadoop jar
>>> >> >> target/giraph-0.70-jar-with-dependencies.jar
>>> >> >> org.apache.giraph.benchmark.PageRankBenchmark -h
>>> >> >> 11/12/13 09:25:08 FATAL conf.Configuration: error parsing conf
>>> >> >> file:
>>> >> >> javax.xml.parsers.ParserConfigurationException: Feature
>>> >> >> 'http://apache.org/xml/features/xinclude' is not recognized.
>>> >> >> Exception in thread "main" java.lang.RuntimeException:
>>> >> >> javax.xml.parsers.ParserConfigurationException: Feature
>>> >> >> 'http://apache.org/xml/features/xinclude' is not recognized.
>>> >> >>         at
>>> >> >>
>>> >> >>
>>> >> >> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1396)
>>> >> >>         at
>>> >> >>
>>> >> >>
>>> >> >> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1251)
>>> >> >>         at
>>> >> >>
>>> >> >> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1192)
>>> >> >>         at
>>> >> >> org.apache.hadoop.conf.Configuration.get(Configuration.java:415)
>>> >> >>         at org.apache.hadoop.util.RunJar.main(RunJar.java:143)
>>> >> >> Caused by: javax.xml.parsers.ParserConfigurationException:
Feature
>>> >> >> 'http://apache.org/xml/features/xinclude' is not recognized.
>>> >> >>         at
>>> >> >>
>>> >> >>
>>> >> >> org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown
>>> >> >> Source)
>>> >> >>         at
>>> >> >>
>>> >> >>
>>> >> >> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1282)
>>> >> >>         ... 4 more
>>> >> >>
>>> >> >> What am I missing?
>>> >> >>
>>> >> >> Thank you,
>>> >> >>
>>> >> >> Shaunak
>>> >> >>
>>> >> >
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> "Now the hardness of this world slowly grinds your dreams away /
>>> >> Makin' a fool's joke out of the promises we make" --- Bruce
>>> >> Springsteen, "Blood Brothers"
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > "Now the hardness of this world slowly grinds your dreams away / Makin'
>>> > a
>>> > fool's joke out of the promises we make" --- Bruce Springsteen, "Blood
>>> > Brothers"
>>
>>
>>
>>
>> --
>> "Now the hardness of this world slowly grinds your dreams away / Makin' a
>> fool's joke out of the promises we make" --- Bruce Springsteen, "Blood
>> Brothers"
>
>
>
>
> --
> "Now the hardness of this world slowly grinds your dreams away / Makin' a
> fool's joke out of the promises we make" --- Bruce Springsteen, "Blood
> Brothers"

Mime
View raw message