giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Asplund <alexaspl...@gmail.com>
Subject Giraph offloadPartition fails creation directory
Date Wed, 11 Sep 2013 22:54:20 GMT
Hi,

I'm still trying to get Giraph to work on a graph that requires more
memory that is available. The problem is that when the Workers try to
offload partitions, the offloading fails. The DiskBackedPartitionStore
fails to create the directory
_bsp/_partitions/job-xxxx/part-vertices-xxx (roughly from recall).

The input or computation will then continue for a while, which I
believe is because it is still managing to hold everything in memory -
but at some point it reaches the limit where there simply is no more
heap space, and it crashes with OOM.

Has anybody had this problem with giraph failing to make HDFS directories?

Mime
View raw message