Forgot about sink processors; yes, it will work.

<= /div>

The trick of this method is you will use a different sink f= or each endpoint, where as the RpcClient (when exposed) will do it all in i= tself. Your configuration will need to look something like this:

-----------------

a1.channels =3D c1

<= channel setup>

a1.sinks =3D k1 k2

a1.sinks.k1.type =3D AVRO

< set up centralFlu= meE connection >

a1.sinks.k1.channel =3D c1

a1.sinks.k2.type =3D AVRO

< set= up centralFlumeF connection >

a1.sinks.k2.channel =3D c1

a1.sinkgroups =3D g1
a1.si= nkgroups.g1.sinks =3D k1 k2
a1.sinkgroups.g1.processor.type =3D load_bal= ance
a1.sinkgroups.g1.processor.backoff =3D true
a1.sinkgroups.g1.pro= cessor.selector =3D round_robin

-----------------

here i= s the relevant link for the load balancing processor:=A0http://f= lume.apache.org/FlumeUserGuide.html#load-balancing-sink-processor

Remember that all sinks in a sink group mus= t share the same channel. This is load balancing, which is what you are see= king in your scenario; the load balancer is not for failover (in the setup = of primary and backup servers), although there is a FailoverSinkProcessor f= or if that's needed.

- Connor

On Wed, Jan 9, 2013 at 11:55 PM, Den= ny Ye <dennyy99@gmail.com> wrote:

hi Hari,=A0
=A0 =A0 I c= annot judge the situation that using method you raised.=A0I would like to e= xplain my case and need your comments. Thanks a lot!

=A0 =A0 What I need is load balancing while event transferring. =A0Ass= ume that I have single local Flume server (located with application) named = 'localFlumeA', configured with single AvroSink and Channel. Meanwhi= le, two central Flume servers (collectors) named 'centralFlumeE' an= d 'centralFlumeF'. Under this case, I would like to configure load = balancing between 'centralFlumeE' and 'centralFlumeF' for e= vents coming from 'localFlumeA', and load can be dispatched average= ly for that two central Flume servers.=A0

=A0 =A0 Can it be configured by LoadBalancingSinkProcessor in your min= d? Wish your advice

-Regards
Denny Ye

2013/1/10 Hari Shreedharan <hshreedharan@cloudera.com>

The LoadBalancing capability similar to the LoadBalanci= ngRpcClient can be configured for multiple Avro Sinks using a LoadBalancing= SinkProcessor, if you are looking for that functionality.

Hari

--=A0
Hari Shreedharan

=20
On Wednesday, January 9, 2013 at= 11:05 PM, Connor Woodson wrote:

Short answer: the= re is no way in the current AvroSink to configure the RpcClient, limiting y= ou to just a single host connection (I'm not sure how well it recovers = if that host goes down).

The AvroSink is incredibly simplified from what the RPCClien= t can do and exposes none of the background functionality. Right now, the o= nly way around that is to create a custom sink based off of the AvroSink so= urce code and instead of setting the RPCClient up the way it currently is, = you pass into the RPCClient.getInstance() a set of user supplied properties= . To implement this in an unsafe way (not checking any of the user's va= lues) would only take a couple lines of code I believe. It is a work around= , but it will enable all of the various RPCClient capabilities such as fail= over or loadbalancing mode and allow it to connect to multiple hosts.

This is something that (I think) there is a JIRA filed for; = but if not, it would be very helpful for this to be implemented into the ac= tual AvroSink (and something that should be linked to that is RPCClient.get= Instance accepting a Context object, simply for ease of use).

- Connor

On Wed, Jan 9, 20= 13 at 10:55 PM, Denny Ye <dennyy99@gmail.com> wrote:

hi all,=A0
=A0 =A0 I didn't find the relation= ship between AvroSink and other types of RpcClient, including LoadBalancing= RpcClient. In my opinion, user can set the specified RpcClient type from Av= roSink with several strategies and host selectors. Also, I cannot get infor= mation from source code and user guide. Did I miss something about this?=A0=

=A0 =A0 =A0Wish someone can support, thanks!

= -Regards
Denny Ye

=20 =20 =20 =20

=20