giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vincentius Martin <vincentiusmar...@gmail.com>
Subject storeCheckpoint() in worker can be slow when a slow worker is presence
Date Sun, 09 Nov 2014 06:39:17 GMT
Hi all,

I have a question related to my last experience using Giraph.

In Giraph worker's code, I see a line like this:

*"getServerData().getCurrentMessageStore().writePartition(verticesOutputStream,
partition.getId());*"

To the best of my knowledge, while executing this line, a worker writes
some of its partitionMap entry into outputStream. However, with the
existence of a slow workers, this line execution in another worker also
gets slower.

I see that the execution gets faster again after the slow worker has
finished its storeCheckpoint process.

>From what I know, each worker uses its own message store and write to
different output stream. Hence, why the slow storeCheckpoint process in a
worker can affect another worker checkpoint process?

Thanks

Mime
View raw message