giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Avery Ching (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-234) Error: GC overhead limit exceeded. Child Error org.apache.hadoop.mapred.TaskRunner.run
Date Thu, 05 Jul 2012 06:53:35 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406882#comment-13406882
] 

Avery Ching commented on GIRAPH-234:
------------------------------------

Jan's suggestion can also be used at runtime i.e. -Dmapred.child.java.opts=-Xmx4000m .  I
usually set -Xms as well to ensure I get the memory.
                
> Error: GC overhead limit exceeded. Child Error org.apache.hadoop.mapred.TaskRunner.run
> --------------------------------------------------------------------------------------
>
>                 Key: GIRAPH-234
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-234
>             Project: Giraph
>          Issue Type: Bug
>          Components: bsp, examples, mapreduce
>    Affects Versions: 0.1.0
>            Reporter: Amani 
>              Labels: example, giraph
>
> Hello everyone, 
> I'm trying to run pregel style implementation of strongly connected components algorithm
on top of giraph. A user defined writable (LongPairWritable) used as the type of the messages
between the vertices. The algorithm works well in single node. However, whenever I tested
using large graph (around 1 million vertices) in cluster, a GC overhead limit exceeded caused
by out of memory exception. The error comes after attempt=0 super-step=1 with 2 minutes pending.
I'm wondering how to solve the problem and what causes it? 
> Is it possible that LongPairWritable cause the problem? 
> I tried a simple test: 
> Succeeded job:
> Comput(Iterator<LongPairWritable> msgIterator){
> If(getSuperstep() >=20){
> voteToHalt();
> } else {
> //Do nothing
> }
> } 
> Failed job:
> Comput(Iterator<LongPairWritable> msgIterator){
> If(getSuperstep() >=20){
> voteToHalt();
> } else {
> sendToAll(new LongPairWritable(new LongWritable(0),new LongWritable(0));
> }
> }
> Thanks in advance. 
> Amani

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message