Hi,

I was able to run the shortest paths example as described here: https://cwiki.apache.org/confluence/display/GIRAPH/Shortest+Paths+Example.

After this I tried to create my own JAR containing a class similar to SimpleShortestPathsVertex and a "lib" directory containing the giraph-*.jar (obtained from the target/ directory after building from source) and other dependent JARs. When I tried to run my JAR as a hadoop job, I get the following errors:

$ hadoop jar giraph-example-shortest-path-with-libs.jar com.skyboximaging.janus.giraph.example.SkyboxShortestPathsVertex shortestPathsInputGraph shortestPathsOutputGraph 0 3

12/02/01 11:05:01 INFO mapred.JobClient: Running job: job_201111161517_1471
12/02/01 11:05:02 INFO mapred.JobClient:  map 0% reduce 0%
12/02/01 11:05:07 INFO mapred.JobClient: Task Id : attempt_201111161517_1471_m_000001_0, Status : FAILED
java.lang.IllegalStateException: run: Caught an unrecoverable exception java.lang.RuntimeException: java.lang.ClassNotFoundException: com.skyboximaging.janus.giraph.example.SkyboxShortestPathsVertex
        at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:650)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
        at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: com.skyboximaging.janus.giraph.example.SkyboxShortestPathsVertex
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1028)
        at org.apache.giraph.graph.BspUtils.getVertexClass(BspUtils.java:299)

When I checked, the class does exist in my JAR, in the right path:

$ jar tvf giraph-example-shortest-path-with-libs.jar | grep SkyboxShortestPathsVertex
  6029 Tue Jan 31 13:22:50 PST 2012 com/skyboximaging/janus/giraph/example/SkyboxShortestPathsVertex.class

Maybe this is yet another Hadoop issue (i.e. not specific to Giraph) but I'd appreciate any help with it.

Thank you,

Shaunak

On Wed, Dec 14, 2011 at 6:57 AM, Shaunak Kashyap <ycombinator@gmail.com> wrote:
Hi Avery,

You're right; my error turned out to be a Hadoop issue which I have
worked around now. Thanks for the official documentation link.

Shaunak

On Tue, Dec 13, 2011 at 5:07 PM, Avery Ching <aching@apache.org> wrote:
> Hi Shaunak,
>
> The official GIRAPH documentation is located at
> https://cwiki.apache.org/confluence/display/GIRAPH/Index .  That being said,
> your error looks like a Hadoop issue.  Are you able to run normal Hadoop
> example jobs?
>
> Thanks,
>
> Avery
>
>
> On 12/13/11 9:27 AM, Shaunak Kashyap wrote:
>>
>> Hi,
>>
>> Maybe this page -
>> https://github.com/aching/Giraph/wiki/Quick-Start-Guide - is outdated
>> but I tried following the instructions there and am getting the
>> following error:
>>
>> shaunak@hadoop-1:~/Giraph$ hadoop jar
>> target/giraph-0.70-jar-with-dependencies.jar
>> org.apache.giraph.benchmark.PageRankBenchmark -h
>> 11/12/13 09:25:08 FATAL conf.Configuration: error parsing conf file:
>> javax.xml.parsers.ParserConfigurationException: Feature
>> 'http://apache.org/xml/features/xinclude' is not recognized.
>> Exception in thread "main" java.lang.RuntimeException:
>> javax.xml.parsers.ParserConfigurationException: Feature
>> 'http://apache.org/xml/features/xinclude' is not recognized.
>>         at
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1396)
>>         at
>> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1251)
>>         at
>> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1192)
>>         at
>> org.apache.hadoop.conf.Configuration.get(Configuration.java:415)
>>         at org.apache.hadoop.util.RunJar.main(RunJar.java:143)
>> Caused by: javax.xml.parsers.ParserConfigurationException: Feature
>> 'http://apache.org/xml/features/xinclude' is not recognized.
>>         at
>> org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown
>> Source)
>>         at
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1282)
>>         ... 4 more
>>
>> What am I missing?
>>
>> Thank you,
>>
>> Shaunak
>>
>



--
"Now the hardness of this world slowly grinds your dreams away /
Makin' a fool's joke out of the promises we make" --- Bruce
Springsteen, "Blood Brothers"



--
"Now the hardness of this world slowly grinds your dreams away / Makin' a fool's joke out of the promises we make" --- Bruce Springsteen, "Blood Brothers"