storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roshan Naik <roshan_n...@yahoo.com>
Subject Re: Computing available parallelism units for storm executors
Date Wed, 10 Apr 2019 02:06:31 GMT
Yes that is basically right. If indeed your executors  are processing enough data to be utilizing
nearly all of cpu then you should consider 1 executor per core. May be a good idea to budget
10-20% cpu per machine for other stuff including interactive login. 


Sent from Yahoo Mail for iPhone


On Tuesday, April 9, 2019, 2:25 PM, Jayant Sharma <sharmajayant27@gmail.com> wrote:

Hi,

I was going through P. Taylor Goetz's talk on scaling storm cluster (2014). He made an interesting
point of computing available parallelism units in a cluster. For CPU bound applications this
limit was 1 executor/CPU core. I just wanted to be sure my understanding about this is clear:
If I have 3 supervisor machines, each having 5 workers or JVMs and each machine has 16 CPU
cores. I have 3*16 = 48 parallelism units to distribute among all of my topologies? Which
means the sum of all spouts and bolts executors across all the topologies should be 48. What
are the implications if I keep my executors more or less than this value?
If my understanding is incorrect, can someone please explain how to compute parallelism units
and relate them to number of executors.
Thanks,Jayant Sharma



Mime
View raw message