giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Dudinski <denis.dudin...@gmail.com>
Subject OutOfCore and checkpointing
Date Tue, 06 Dec 2016 07:43:33 GMT
Hello,

I'm trying to run graph computation with OutOfCore and checkpointing
enabled. Checkpoints are created in hdfs as expected. But when I try to
restart computation from one of these checkpoints computation fails with
OutOfMemory. It seems that all partitions are loaded into memory during
checkpoint restore step of computation which leads to memory starvation
(OOC is used to prevent this). I use FixedPartitionsOracle as oracle for
OOC.

Is it possible to use checkpointing with OOC?

Thank you in advance.

Best regards,
Denis

Mime
View raw message