giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Asplund <alexaspl...@gmail.com>
Subject Re: Giraph offloadPartition fails creation directory
Date Thu, 12 Sep 2013 20:57:50 GMT
Unfortunately there's some restrictions that means I don't really have
them handy, BUT pointing me towards the local disk helped me partially
resolve it. There are rights issues with this directory, but I was
able to get by it by manually creating a separate giraph in mapreduce
local storage mapred/local/giraph and setting Giraph Options to point
to <local storage>/giraph/partitions and /messages

Then something strange happens. The job successfully creates 30,
exactly 30 directories, and then starts failing again. This happened
both times I ran the job. 30 directories are created in the partitions
directory, and then subsequenctly it prints to the task log something
like

DiskBackedPartitionStorage: offloadPartition: Failed to create directory <...>

..and then no further directories are created after the 30. It will
attempt to create more partition directories, but it keeps failling
after the inital 30. It is quite strange.

On 9/12/13, Claudio Martella <claudio.martella@gmail.com> wrote:
> Giraph does not offload partitions or messages to HDFS in the out-of-core
> module. It uses local disk on the computing nodes. By defualt, it uses the
> tasktracker local directory where for example the distributed cache is
> stored.
>
> Could you provide the stacktrace Giraph is spitting when failing?
>
>
> On Thu, Sep 12, 2013 at 12:54 AM, Alexander Asplund
> <alexasplund@gmail.com>wrote:
>
>> Hi,
>>
>> I'm still trying to get Giraph to work on a graph that requires more
>> memory that is available. The problem is that when the Workers try to
>> offload partitions, the offloading fails. The DiskBackedPartitionStore
>> fails to create the directory
>> _bsp/_partitions/job-xxxx/part-vertices-xxx (roughly from recall).
>>
>> The input or computation will then continue for a while, which I
>> believe is because it is still managing to hold everything in memory -
>> but at some point it reaches the limit where there simply is no more
>> heap space, and it crashes with OOM.
>>
>> Has anybody had this problem with giraph failing to make HDFS
>> directories?
>>
>
>
>
> --
>    Claudio Martella
>    claudio.martella@gmail.com
>


-- 
Alexander Asplund

Mime
View raw message