giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amani (JIRA)" <j...@apache.org>
Subject [jira] [Created] (GIRAPH-234) Error: GC overhead limit exceeded. Child Error org.apache.hadoop.mapred.TaskRunner.run
Date Wed, 04 Jul 2012 12:20:36 GMT
Amani  created GIRAPH-234:
-----------------------------

             Summary: Error: GC overhead limit exceeded. Child Error org.apache.hadoop.mapred.TaskRunner.run
                 Key: GIRAPH-234
                 URL: https://issues.apache.org/jira/browse/GIRAPH-234
             Project: Giraph
          Issue Type: Bug
          Components: bsp, examples, mapreduce
    Affects Versions: 0.1.0
            Reporter: Amani 


Hello everyone, 

I'm trying to run pregel style implementation of strongly connected components algorithm on
top of giraph. A user defined writable (LongPairWritable) used as the type of the messages
between the vertices. The algorithm works well in single node. However, whenever I tested
using large graph (around 1 million vertices) in cluster, a GC overhead limit exceeded caused
by out of memory exception. The error comes after attempt=0 super-step=1 with 2 minutes pending.
I'm wondering how to solve the problem and what causes it? 

Is it possible that LongPairWritable cause the problem? 
I tried a simple test: 
Succeeded job:
Comput(Iterator<LongPairWritable> msgIterator){
If(getSuperstep() >=20){
voteToHalt();
} else {
//Do nothing
}
} 

Failed job:
Comput(Iterator<LongPairWritable> msgIterator){
If(getSuperstep() >=20){
voteToHalt();
} else {
sendToAll(new LongPairWritable(new LongWritable(0),new LongWritable(0));
}
}

Thanks in advance. 

Amani



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message