flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abhijeet Pathak <Abhijeet.Pat...@kpitcummins.com>
Subject RE: Configuring failover sync in Flume-ng
Date Thu, 20 Dec 2012 07:48:53 GMT
Okay.

-----Original Message-----
From: Brock Noland [mailto:brock@cloudera.com] 
Sent: Wednesday, December 19, 2012 8:10 PM
To: user@flume.apache.org
Subject: Re: Configuring failover sync in Flume-ng

Just as a side note: File Channel has received significantly more testing than JDBC Channel.

On Wed, Dec 19, 2012 at 8:05 AM, Alexander Alten-Lorenz <wget.null@gmail.com> wrote:
> Yes, why not luring the wiki :)
> https://cwiki.apache.org/confluence/display/FLUME/Home
>
> Stable: http://www.apache.org/dyn/closer.cgi/flume/1.3.0
> Just grab the tar.gz - or use 1.2.0 as deb-package, I suspect the balancing stuff is
inside too.
>
> Best,
>  Alex
>
> On Dec 19, 2012, at 2:55 PM, Abhijeet Pathak <Abhijeet.Pathak@kpitcummins.com>
wrote:
>
>> I am running Ubuntu 12.04 server edition.
>> The repo has flume 1.2 release available.
>> How should I install 1.3? Using source?
>> Is there a stable release available?
>>
>> Thanks,
>> Abhijeet
>>
>> -----Original Message-----
>> From: Alexander Alten-Lorenz [mailto:wget.null@gmail.com]
>> Sent: Wednesday, December 19, 2012 4:33 PM
>> To: user@flume.apache.org
>> Subject: Re: Configuring failover sync in Flume-ng
>>
>> Oh, yes. You should upgrade into 1.3.0, which was released last week. 
>> Wasn't clear for me that you're running on 1.1.0
>>
>> Do you have a change to upgrade into the newest release?
>>
>> Best,
>> Alex
>>
>> On Dec 19, 2012, at 11:51 AM, Abhijeet Pathak <Abhijeet.Pathak@kpitcummins.com>
wrote:
>>
>>> It's still not working.
>>>
>>> I read somewhere that failover support was implemented after Jan 2012.
>>> I've flume 1.1.0+121-1.cdh4.0.1.p0.1~precise-cdh4.0.1 installed.
>>>
>>> Can that be a reason for it not working?
>>>
>>> Thanks,
>>> Abhijeet
>>>
>>> -----Original Message-----
>>> From: Alexander Alten-Lorenz [mailto:wget.null@gmail.com]
>>> Sent: Wednesday, December 19, 2012 2:52 PM
>>> To: user@flume.apache.org
>>> Subject: Re: Configuring failover sync in Flume-ng
>>>
>>> YOUR_NAME.sinks = HOSTSINK-1 HOSTSINK-2
>>>
>>> HOSTSINK-1.type=
>>> HOSTSINK-1.bind=HOSTNAME-1
>>> HOSTSINK-1.port=needed by JDBC?
>>> HOSTSINK-1.channels=YOUR_CHANNELS
>>>
>>> the same for the other HOSTSINK and deploy the config to the hosts and startup.
Should work.
>>>
>>> Best,
>>> Alex
>>>
>>> On Dec 19, 2012, at 10:01 AM, Abhijeet Pathak <Abhijeet.Pathak@kpitcummins.com>
wrote:
>>>
>>>> Yes, I saw that.
>>>>
>>>> But I'm not sure how to tell flume that the failover sink is running on "another"
machine.
>>>>
>>>> -----Original Message-----
>>>> From: Alexander Lorenz [mailto:wget.null@gmail.com]
>>>> Sent: Wednesday, December 19, 2012 1:59 PM
>>>> To: user@flume.apache.org
>>>> Subject: Re: Configuring failover sync in Flume-ng
>>>>
>>>> Hi,
>>>>
>>>> Failover:
>>>> http://flume.apache.org/FlumeUserGuide.html#failover-sink-processor
>>>>
>>>> Just define more as one JDBC channel and add them to the failover config
(see the examples).
>>>>
>>>> Best
>>>> - Alex
>>>>
>>>> --
>>>> Alexander Alten-Lorenz
>>>> http://mapredit.blogspot.com
>>>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>>>
>>>> On Dec 19, 2012, at 9:23 AM, Abhijeet Pathak <Abhijeet.Pathak@kpitcummins.com>
wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc],
1 sink [AsyncHbase]) on one server.
>>>>>
>>>>> However, I am not able to find documentation on how to configure failover
sinks for such scenario.
>>>>>
>>>>> I want to run same sink as backup/failover sink on different server.
>>>>> How can that be configured?
>>>>>
>>>>> Thanks,
>>>>> Abhijeet
>>>>>
>>>>
>>>
>>
>



--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/



Mime
View raw message