storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rudraneel chakraborty <rudraneel.chakrabo...@gmail.com>
Subject Re: Need Help regarding topology with numWorker>1
Date Fri, 12 Feb 2016 22:42:15 GMT
Thanks. Actually i have to create a scenario where 2 worker performs better
than one worker. But in reality , topology with a single worker performs
considerably better.

I sending csv lines to kafka (5 partitions ) and reading them from a
topology with kafka spout (parallelism hint 5)

Any thoughts?

On Thursday, 11 February 2016, Nathan Leung <ncleung@gmail.com> wrote:

> Any situation where you require more CPU than 1 server can provide for you
> - there are tuning parameters (e.g. localOrShuffleGrouping) that you can
> use to reduce the amount of data sent over the network too.
>
> Any situation where you need to have tolerance in case of machine failure.
>
>
>
> On Thu, Feb 11, 2016 at 2:58 PM, Rudraneel chakraborty <
> rudraneel.chakraborty@gmail.com
> <javascript:_e(%7B%7D,'cvml','rudraneel.chakraborty@gmail.com');>> wrote:
>
>> Topology param numWorkers i meant
>>
>>
>> On Thursday, 11 February 2016, Rudraneel chakraborty <
>> rudraneel.chakraborty@gmail.com
>> <javascript:_e(%7B%7D,'cvml','rudraneel.chakraborty@gmail.com');>> wrote:
>>
>>> More specifically , i have seen a topology performs better if it is
>>> assigned a single worker compared to more than one worker.
>>>
>>> I want a situation where a topology performs better with more than one
>>> worker.
>>>
>>> And it doesnt matter if both workers are on same supervisor or different
>>> supervisor
>>>
>>> On Thursday, 11 February 2016, Matthias J. Sax <mjsax@apache.org> wrote:
>>>
>>>> I am not sure what you mean:
>>>>
>>>>   - number of worker slots per supervisor
>>>> or
>>>>   - topology parameter "number of workers"
>>>>
>>>> Can you clarify?
>>>>
>>>> -Matthias
>>>>
>>>> On 02/11/2016 05:14 AM, anshu shukla wrote:
>>>> > Not  like that..  But i have used workers equal to number of cores.
>>>> Each
>>>> > vm with 8 corea.
>>>> >
>>>> > On 11 Feb 2016 9:07 am, "Rudraneel chakraborty"
>>>> > <rudraneel.chakraborty@gmail.com
>>>> > <mailto:rudraneel.chakraborty@gmail.com>> wrote:
>>>> >
>>>> >     more than one worker on same node ? Did u use custom schedule r?
>>>> >     because by default, the workers would be spread throughout the
>>>> cluster
>>>> >
>>>>
>>>>
>>>
>>> --
>>> Rudraneel Chakraborty
>>> Carleton University Real Time and Distributed Systems Reserach
>>>
>>>
>>
>> --
>> Rudraneel Chakraborty
>> Carleton University Real Time and Distributed Systems Reserach
>>
>>
>

-- 
Rudraneel Chakraborty
Carleton University Real Time and Distributed Systems Reserach

Mime
View raw message