incubator-flume-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hari Shreedharan (Updated) (JIRA)" <>
Subject [jira] [Updated] (FLUME-1053) Basic SourceConfiguration
Date Tue, 27 Mar 2012 06:03:33 GMT


Hari Shreedharan updated FLUME-1053:

    Description:     (was: Currently the configuration of each component sits deep inside
the component themselves. There is no way to verify if a configuration is valid before run
time. In most systems like Flume, it is likely that these confs will be deployed from one
single host, on to the machines where flume agents are running. Only when each agent starts,
invalid confs are identified because the Agents would terminate by throwing exceptions. This
is a first attempt to make a component-aware configuration system which is independent of
the Flume, and does not require the Flume jar to be installed. Each component needs a configuration
manager, which configures the components. 

Provide abstract Configuration stubs for each component type, sources, channels, sinks, selectors
etc, which are in the new package, independent on ng-core. Now for each of the channels extend
the abstract class and check the config properties for each of the required parameters for
that component, for example: MultiplexingChannelSelectorConfigurator would look for default
channel etc. If a particular component does not have a configuration class then let the current
mechanism continue. 

This will require implementation for each component, but it should not be too complex. One
additional advantage we get from this is that we can separate out the config validation from
the components into these stubs, but we will still need to read the values out of the Context
as we do currently(else we end up making the configuration dependent on flume-ng-core itself
which we wanted to avoid). 

A problem with this is making a change to the configuration would require changes in the configuration
classes and in the components also(where the configuration is read and the component is actually
configured). I could not figure out a way of avoiding this.
        Summary: Basic SourceConfiguration  (was: CLONE - Create configuration stubs for sources,
channels, sinks etc)
> Basic SourceConfiguration
> -------------------------
>                 Key: FLUME-1053
>                 URL:
>             Project: Flume
>          Issue Type: Sub-task
>    Affects Versions: v1.0.0
>            Reporter: Hari Shreedharan
>            Assignee: Hari Shreedharan
>             Fix For: v1.2.0

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message