giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hassan Eslami <hsn.esl...@gmail.com>
Subject Re: OutOfCore and checkpointing
Date Tue, 06 Dec 2016 16:16:25 GMT
Hi Denis,

OOC + Checkpointing is one of the TODO's in the code base at the moment.
Unfortunately, OOC cannot be used in combination with checkpointing at the
moment. We are planning to make these two work with each other in the
future.

Best,
Hassan

On Tue, Dec 6, 2016 at 1:43 AM, Denis Dudinski <denis.dudinski@gmail.com>
wrote:

> Hello,
>
> I'm trying to run graph computation with OutOfCore and checkpointing
> enabled. Checkpoints are created in hdfs as expected. But when I try to
> restart computation from one of these checkpoints computation fails with
> OutOfMemory. It seems that all partitions are loaded into memory during
> checkpoint restore step of computation which leads to memory starvation
> (OOC is used to prevent this). I use FixedPartitionsOracle as oracle for
> OOC.
>
> Is it possible to use checkpointing with OOC?
>
> Thank you in advance.
>
> Best regards,
> Denis
>

Mime
View raw message