flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guillermo Ortiz <konstt2...@gmail.com>
Subject Re: Load Balancing.
Date Thu, 11 Sep 2014 11:53:35 GMT
I guess that there're many open files open in the same time. What means
that each event it's delivered some sink randomly.

2014-09-11 10:27 GMT+02:00 Guillermo Ortiz <konstt2000@gmail.com>:

> I checked, you're right, is there more documentation about it somewhere?
> How does it work? I mean, if I choose random, it choose randomly a sink
> and when it finishes to generate a file and choose another sink or it send
> each event for each sink and really there are many sink generating files
> simultaneously?
>
>
> 2014-09-11 3:08 GMT+02:00 Ashish <paliwalashish@gmail.com>:
>
>> IMHO,
>> http://flume.apache.org/FlumeUserGuide.html#load-balancing-sink-processor
>> explains it well. Failover Sink processor is used in case of failure event,
>> but Load balancing Sink processor would be used to choose the Sink for
>> sending Events.
>>
>> Can try this with a simple topology with two File Roll sinks, configured
>> to use Load Balancing Sink Processor.
>>
>> On Wed, Sep 10, 2014 at 9:22 PM, Guillermo Ortiz <konstt2000@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I want to use the load balancer in Flume. After reading Flume
>>> documentation about loading balancing I think that it's not really load
>>> balacing, if I understood well, when a sink fail, it chooses a new sink
>>> with ronud-robin or in a random way. But it doesn't distributed on this way
>>> all the time among the differents sink.
>>>
>>> Is there any possiblity of distributed the load among all the sinks?
>>> Because if I undertood well, the load balancing option doesn't do it.
>>>
>>>
>>
>>
>>
>> --
>> thanks
>> ashish
>>
>> Blog: http://www.ashishpaliwal.com/blog
>> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>>
>
>

Mime
View raw message