storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alok Kumbhare <kumbh...@usc.edu>
Subject Storm Parallelism/Scaling
Date Wed, 16 Jul 2014 00:36:50 GMT
Hi,
I have a few questions regarding the parallelism in Storm.

I have gone through the documentation at
http://storm.incubator.apache.org/documentation/Understanding-the-parallelism-of-a-Storm-topology.html
but I have some confusion regarding executors and tasks.

>From what I understand (to simplify, assume there is only one topology
running):

Worker processes runs executor threads (one or more executor per worker).
Since each executor is an independent thread, they run in parallel.

Now the documentation says "each Executor, runs one or more task of same
component". My question is whether the tasks run in parallel or in
sequence? What does it mean to have multiple tasks of the same component
per executor?


Further, the documentation mentions: "The number of tasks for a component
is always the same throughout the lifetime of a topology, but the number of
executors (threads) for a component can change over time."

I don't understand this part. Assuming the default configuration where each
executor runs one task (i.e. number of tasks is equal to the number of
executors) and I run the topology with some initial configuration. Now, if
I use the "rebalance" command and increase the number of executors, what
happens to the number of tasks?

Please help.

Thanks,
Alok

Mime
View raw message