flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashish <paliwalash...@gmail.com>
Subject Re: Flume dynamic configuraiton
Date Fri, 01 Aug 2014 11:16:23 GMT
On Fri, Aug 1, 2014 at 4:40 PM, Paweł <prog88@gmail.com> wrote:

> Hi,
> Thanks for information. Please tell me what happens with existing sources,
> channels and sinks when configuration is reloaded? Are they destroyed and
> started once again?
>

During reload Flume shall stop existing ones and the reload the config
creating Components as specified in changed config.

See org.apache.flume.lifecycle package for Life cycle stuff
and org.apache.flume.node package for how Agent starts and use
Configuration.


>
> Zookeeper would be nice here. Can you tell me if there are any estimates
> for time needed to close the issue or to have any stable version (no needed
> to be pushed into master)?
>

Patch is attached, need to be battle tested.


>
> --
> Paweł
>
>
> 2014-08-01 12:59 GMT+02:00 Ashish <paliwalashish@gmail.com>:
>
> If you change the configuration file, Flume reloads it, unless you tell it
>> not to by specifying no-reload-conf option.
>>
>>
>> On Fri, Aug 1, 2014 at 3:35 PM, Paweł <prog88@gmail.com> wrote:
>>
>>> Hi,
>>> Currently I see no option to configure Flume dynamically.
>>>
>>> I have a case where I have to be able to
>>> * add or remove Source
>>> * attach Source to a Channel
>>> * attach a Channel to a Sink or several Sinks
>>>
>>> All of this things are possible but only with configuration file which
>>> means restart of application each time when the anything changes.
>>>
>>> 1) Are there any plans to add dynamic configuration mechanisms?
>>>
>>
>> I think it's already supported. When you reconfigure, Flume Agents goes
>> through its Lifecycle events stop and then restart.
>>
>>
>>> 2) Do you think any REST API or JMX would be good in this place?
>>>
>>
>> There is JIRA for configuration using Zookeeper
>> https://issues.apache.org/jira/browse/FLUME-1491
>>
>>
>>> 3) How complicated it is to make Flume configurable? Does it men tons of
>>> coding and months to implement it or rather it is not so hard?
>>>
>>
>>
>>>
>>> --
>>> Paweł
>>>
>>
>>
>>
>> --
>> thanks
>> ashish
>>
>> Blog: http://www.ashishpaliwal.com/blog
>> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>>
>
>


-- 
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal

Mime
View raw message