giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Krause ...@ckrause.org>
Subject Optimal configuration for benchmark
Date Thu, 27 Jun 2013 16:20:38 GMT
Hi,

I implemented a benchmark that allows me to generate an arbitrarily 
large graph (depending on the number of iterations). Now I would like to 
configure Giraph so that I can make the best use of my hardware for this 
benchmark. Based on the number of nodes in my cluster, their amount of 
main memory and number of cores, I am asking myself how do I determine 
the optimal parameters of Giraph / Hadoop, specifically:

- the number of used mappers
- the HEAP_SIZE environment variable
- the memory specified in the mapred.map.child.java.opts property

(any other relevant parameters?)

Also, I was wondering how well Giraph can handle computations which 
start with a very small graph and mutate it to a very large one. For 
example, if I understand correctly the number of mappers is not 
dynamically adjusted.

Any hints (or links to documentation) are highly appreciated.

Cheers,
Christian


Mime
View raw message