flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roshan Naik <ros...@hortonworks.com>
Subject Re: Failover in Flume NG
Date Tue, 16 Oct 2012 17:32:49 GMT
There is support for failover and load balancing. Take a look at sink
processors here
http://flume.apache.org/FlumeUserGuide.html#flume-sink-processors
You could have failover sinks configured to send it a different agent(i,e
collector).

-roshan


On Tue, Oct 16, 2012 at 10:09 AM, Pankaj Gupta <pankaj@brightroll.com>wrote:

> Hi,
>
>
> We are investigating using Flume for log aggregation in our organization
> and have this dilemma over going with Flume NG or OG. We really like the
> centralized configuration management in OG but that's not a must have. What
> we need most is reliability in face of machines going down (which is quite
> common on AWS). How is that handled in Flume NG. In OG, I believe, it was
> possible to define failover semantics so that messages could be routed to
> another collector if one collector went down. In Flume OG there is no
> concept of a special collector but collector is just an agent as I
> understand. I'm wondering how failover semantics are defined for these
> agents. It would be great if someone could elaborate on that.
>
> We also want to use a relatively new version of Hadoop e.g. 1.0.3 or
> higher. Is it possible to use Flume NG (or OG) with the new versions of
> Hadoop?
>
> Thanks in advance,
>
> Pankaj
>
> --
>
>
> *P* | (415) 677-9222 ext. 205 *F *| (415) 677-0895 | pankaj@brightroll.com
>
> Pankaj Gupta | Software Engineer
>
> *BrightRoll, Inc. *| Smart Video Advertising | www.brightroll.com
>
>
> United States | Canada | United Kingdom | Germany
>
>
> We're hiring<http://newton.newtonsoftware.com/career/CareerHome.action?clientId=8a42a12b3580e2060135837631485aa7>
> !
>
>

Mime
View raw message