giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Francesco Sclano <>
Subject how determine the number of workers to set in -w argument ?
Date Sun, 02 Sep 2018 20:33:44 GMT
I'm using an ec2 hadoop cluster that is comprised of 20 c3.8xlarge
machines, each having 60 GB RAM and 32 virtual CPUs.
In every machine I set up yarn and mapreduce settings as documented here,
i.e. as showed below:

Configuration Option    Default Value    -Xmx1331m    -Xmx2662m    1664
mapreduce.reduce.memory.mb    3328    3328
yarn.scheduler.minimum-allocation-mb    32
yarn.scheduler.maximum-allocation-mb    53248
yarn.nodemanager.resource.memory-mb    53248

Now what criteria I have to use in order to determine the most appropriate
number of workers to use with giraph? I.e. what number I have to use for -w
argument? Is that criteria related to above settings?

Francesco Sclano

View raw message