storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jayant Sharma <sharmajayan...@gmail.com>
Subject Computing available parallelism units for storm executors
Date Tue, 09 Apr 2019 21:25:24 GMT
Hi,

I was going through P. Taylor Goetz's talk on scaling storm cluster (2014).
He made an interesting point of computing available parallelism units in a
cluster. For CPU bound applications this limit was 1 executor/CPU core. I
just wanted to be sure my understanding about this is clear:

If I have 3 supervisor machines, each having 5 workers or JVMs and each
machine has 16 CPU cores. I have 3*16 = 48 parallelism units to distribute
among all of my topologies? Which means the sum of all spouts and bolts
executors across all the topologies should be 48. What are the implications
if I keep my executors more or less than this value?

If my understanding is incorrect, can someone please explain how to compute
parallelism units and relate them to number of executors.

Thanks,
Jayant Sharma

Mime
View raw message