giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jyoti Yadav <>
Subject Running out of space..
Date Sun, 02 Feb 2014 16:25:42 GMT
Hi folks..

While executing one algorithm with enabling checkpointing feature on,system
is running out of memory. I gave following command

hadoop jar
-Dgiraph.useSuperstepCounters=false -Dgiraph.useOutOfCoreMessages=true
org.apache.giraph.examples.MyShortestDistance -vif
-vip /user/hduser/big_input/clustering -vof -op
/user/hduser/sp_output537/sd_output -w 1 -mc

By executing this command   /app/hadoop/tmp/dfs/name/data/current directory
is consuming lots of space,approx  90GB(or may be more than this)
This job execution is involving thousands of supersteps..If i do not enable
checkpoint feature  on then algorithm fails due to worker failure error
after 19000 around supersteps.

Pls suggest me how to deal with this issue??


View raw message