giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Dudinski <denis.dudin...@gmail.com>
Subject Re: OutOfCore and checkpointing
Date Wed, 07 Dec 2016 13:17:06 GMT
Thanks!

Best Regards,
Denis Dudinski

2016-12-06 19:16 GMT+03:00 Hassan Eslami <hsn.eslami@gmail.com>:

> Hi Denis,
>
> OOC + Checkpointing is one of the TODO's in the code base at the moment.
> Unfortunately, OOC cannot be used in combination with checkpointing at the
> moment. We are planning to make these two work with each other in the
> future.
>
> Best,
> Hassan
>
> On Tue, Dec 6, 2016 at 1:43 AM, Denis Dudinski <denis.dudinski@gmail.com>
> wrote:
>
>> Hello,
>>
>> I'm trying to run graph computation with OutOfCore and checkpointing
>> enabled. Checkpoints are created in hdfs as expected. But when I try to
>> restart computation from one of these checkpoints computation fails with
>> OutOfMemory. It seems that all partitions are loaded into memory during
>> checkpoint restore step of computation which leads to memory starvation
>> (OOC is used to prevent this). I use FixedPartitionsOracle as oracle for
>> OOC.
>>
>> Is it possible to use checkpointing with OOC?
>>
>> Thank you in advance.
>>
>> Best regards,
>> Denis
>>
>
>

Mime
View raw message