giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manuel Lagang <manuellag...@gmail.com>
Subject setWorkerConfiguration parameters clarification
Date Thu, 10 Oct 2013 23:12:51 GMT
I'm trying to understand the meaning of the 3 parameters to
GiraphConfiguration.setWorkerConfiguration: minWorkers, maxWorkers, and
minPercentResponded. I want my Giraph jobs to co-exist nicely with other
jobs in the cluster, and it's not always the case that I can get a fixed
number of map slots for my job before the job times out. Thus, I would like
the job to be able to start with a possibly smaller set of workers, ideally
being able to pick up workers in later supersteps.

So I tried <minWorkers=10,maxWorkers=50,minPercentResponded=100>, expecting
this to mean that it can start with 10 workers provided that 100% of those
workers respond. But this setting ends up again waiting for all 50 workers.

Then I tried <minWorkers=10,maxWorkers=50,minPercentResponded=20>,
expecting that minPercentResponded was just a redundant expression of
minWorkers/maxWorkers. But this setting leads to null pointer exceptions in
org.apache.giraph.comm.SendCache.removeWorkerData(SendCache.java:199).

So I must be confused about the meaning of these variables, and what the
legal values are. Can anyone enlighten me on how (if possible) I can get
the behavior I want?

Thanks,
Manuel Lagang

Mime
View raw message