If your workload does not saturate a single machine, it will of course
be more efficient to run within a single worker as you avoid intra-JVM
communication.
As long as CPU, memory, or network of a single machine is not utilized
completely, you will not benefit from multiple workers from a
performance point of view.
Btw: I am a not sure what you mean by "I have to create a scenario..."?
On 02/12/2016 11:42 PM, Rudraneel chakraborty wrote:
> Thanks. Actually i have to create a scenario where 2 worker performs
> better than one worker. But in reality , topology with a single worker
> performs considerably better.
>
> I sending csv lines to kafka (5 partitions ) and reading them from a
> topology with kafka spout (parallelism hint 5)
>
> Any thoughts?
>
> On Thursday, 11 February 2016, Nathan Leung <ncleung@gmail.com
> <mailto:ncleung@gmail.com>> wrote:
>
> Any situation where you require more CPU than 1 server can provide
> for you - there are tuning parameters (e.g. localOrShuffleGrouping)
> that you can use to reduce the amount of data sent over the network too.
>
> Any situation where you need to have tolerance in case of machine
> failure.
>
>
>
> On Thu, Feb 11, 2016 at 2:58 PM, Rudraneel chakraborty
> <rudraneel.chakraborty@gmail.com
> <javascript:_e(%7B%7D,'cvml','rudraneel.chakraborty@gmail.com');>>
> wrote:
>
> Topology param numWorkers i meant
>
>
> On Thursday, 11 February 2016, Rudraneel chakraborty
> <rudraneel.chakraborty@gmail.com
> <javascript:_e(%7B%7D,'cvml','rudraneel.chakraborty@gmail.com');>>
> wrote:
>
> More specifically , i have seen a topology performs better
> if it is assigned a single worker compared to more than one
> worker.
>
> I want a situation where a topology performs better with
> more than one worker.
>
> And it doesnt matter if both workers are on same supervisor
> or different supervisor
>
> On Thursday, 11 February 2016, Matthias J. Sax
> <mjsax@apache.org> wrote:
>
> I am not sure what you mean:
>
> - number of worker slots per supervisor
> or
> - topology parameter "number of workers"
>
> Can you clarify?
>
> -Matthias
>
> On 02/11/2016 05:14 AM, anshu shukla wrote:
> > Not like that.. But i have used workers equal to
> number of cores. Each
> > vm with 8 corea.
> >
> > On 11 Feb 2016 9:07 am, "Rudraneel chakraborty"
> > <rudraneel.chakraborty@gmail.com
> > <mailto:rudraneel.chakraborty@gmail.com>> wrote:
> >
> > more than one worker on same node ? Did u use
> custom schedule r?
> > because by default, the workers would be spread
> throughout the cluster
> >
>
>
>
> --
> Rudraneel Chakraborty
> Carleton University Real Time and Distributed Systems Reserach
>
>
>
> --
> Rudraneel Chakraborty
> Carleton University Real Time and Distributed Systems Reserach
>
>
>
>
> --
> Rudraneel Chakraborty
> Carleton University Real Time and Distributed Systems Reserach
>
|