storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suchi Amalapurapu <su...@bloomreach.com>
Subject scheduling multiple source spouts
Date Fri, 03 Jan 2014 16:37:57 GMT
All
I have N single topologies each with a single kafka spout(case #1).
Trying to replace this with a single topology with N spouts(case #2).
The rest of the settings are all the same.

The message producers are all the same. Ideally the delta between
production and consumption of messages should be the same in both cases but
its several folds worse in the second case.

Is this due to some scheduling overhead among the N spouts?
If a topology has N spouts, how do tuples get generated? What kind of
scheduling happens here across spouts?
I am using storm 0.8.1 and kafka 0.7.2
Suchi

Mime
View raw message