giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Muaz Twaty <>
Subject Hadoop workers assigning
Date Fri, 19 Apr 2019 08:35:18 GMT
Hello Giraph community,

I recently started working with Giraph and I have question which I did not
find an answer to it. Is there a way in which we can control the way the
workers are assigned to the cluster nodes? The problem that I'm facing is
that sometimes most of the workers tend to be assigned to a small number of
Datanodes while other Datanodes are not running any. This is causing an Out
Of Memory problem. For example, if we are running 3 Datanodes and we have 6
workers, sometimes, 4 workers are assigned to one Datanode and 2 workers
are assigned to another Datanode and the third Datanode is not running any.
My project aiming to find an automated method to optimize Giraph parameters
so I am hoping that controlling this could help my optimization algorithm
and prevent allocating redundant memory.


*Muaz TWATY*

♻ Be green, keep it on the screen

View raw message