giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koontz (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-198) running Giraph trunk on Hadoop 2.0.0-alpha leads to an exception
Date Wed, 13 Jun 2012 15:56:42 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294541#comment-13294541
] 

Eugene Koontz commented on GIRAPH-198:
--------------------------------------

Hi Roman, 

I think the key is the message in your immediately above-quoted logs:

{code}
2012-06-07 17:07:31,033 INFO [org.apache.giraph.graph.MasterThread] org.apache.giraph.graph.BspService:
getApplicationAttempt: Node /_hadoopBsp/job_1339098338323_0011/_applicationAttemptsDir already
exists!
{code}

I see similar messages when running a new Giraph job after a previous Giraph job has failed
: the previous job is not being cleaned up. 

Try to start your Zookeeper client and run: 

{code}
rmr /_hadoopBsp/job_1339098338323_011
{code}

and then rerun your job. Or you could also simply do:
{code}
rmr /_hadoopBsp
{code}

and rerun.

I think we need better cleanup of Zookeeper directories of failed jobs, and better handling
of errors related to already-existing ZK dirs of such jobs, so I'll file a JIRA about that
and relate it to this one.
                
> running Giraph trunk on Hadoop 2.0.0-alpha leads to an exception
> ----------------------------------------------------------------
>
>                 Key: GIRAPH-198
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-198
>             Project: Giraph
>          Issue Type: Bug
>          Components: mapreduce
>    Affects Versions: 0.2.0
>            Reporter: Roman Shaposhnik
>            Assignee: Eugene Koontz
>             Fix For: 0.2.0
>
>
> Here's how we compile:
> {noformat}
> $ mvn -Phadoop_trunk -DskipTests -Dhadoop.version=2.0.0-alpha site package
> {noformat}
> Here's how we run:
> {noformat}
> $ export GIRAPH_HOME=/usr/lib/giraph 
> $ hadoop jar /usr/lib/giraph/giraph-jar-with-dependencies.jar org.apache.giraph.benchmark.PageRankBenchmark
-e 1 -s 3 -V 100000 -w 3
> {noformat}
> Here's what we get in the task logs (as a side note -- once 3 of the 8 tasks fail that
way the entire job hangs forever which could be a bug in and of itself):
> {noformat}
> 2012-05-31 10:31:28,518 WARN [main]
> org.apache.hadoop.mapred.YarnChild: Exception running child :
> java.lang.RuntimeException: java.lang.NoSuchFieldException: versionID
>        at org.apache.hadoop.ipc.RPC.getProtocolVersion(RPC.java:170)
>        at org.apache.hadoop.ipc.WritableRpcEngine$Invocation.(WritableRpcEngine.java:110)
>        at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
>        at $Proxy11.putVertexList(Unknown Source)
>        at org.apache.giraph.comm.BasicRPCCommunications.sendPartitionReq(BasicRPCCommunications.java:921)
>        at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:322)
>        at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:594)
>        at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:473)
>        at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:645)
>        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:725)
>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
>        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at javax.security.auth.Subject.doAs(Subject.java:396)
>        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
>        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
> Caused by: java.lang.NoSuchFieldException: versionID
>        at java.lang.Class.getField(Class.java:1520)
>        at org.apache.hadoop.ipc.RPC.getProtocolVersion(RPC.java:166)
>        ... 15 more
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message