flume-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "darkz (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLUME-946) Allow multiplexing channel selector to specify optional channels.
Date Thu, 07 Sep 2017 02:55:01 GMT

    [ https://issues.apache.org/jira/browse/FLUME-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156363#comment-16156363

darkz commented on FLUME-946:

helpful for me!!

> Allow multiplexing channel selector to specify optional channels.
> -----------------------------------------------------------------
>                 Key: FLUME-946
>                 URL: https://issues.apache.org/jira/browse/FLUME-946
>             Project: Flume
>          Issue Type: Improvement
>    Affects Versions: 1.0.0, 1.1.0
>            Reporter: Arvind Prabhakar
>            Assignee: Hari Shreedharan
>            Priority: Minor
>             Fix For: 1.3.0
>         Attachments: FLUME-946-2.patch, FLUME-946-3.patch
>   Original Estimate: 0h
>  Remaining Estimate: 0h
> Right now the multiplexing channel selector treats all channels as required channels.
There should be a simple provision where some channel can be marked optional so that a failure
in publishing the event to that particular channel does not cause the entire request to fail.
For example, the following configuration:
> agent.sources.foo.selector.mapping.foo = ch1 ch2
> agent.sources.foo.selector.mapping.bar = ch2 ch3
> agent.sources.foo.selector.optional.channels = ch2
> Would imply that for mapping value "foo", ch1 is a required channel whereas ch2 is an
optional channel. Similarly, for mapping "bar', ch3 is a required channel but ch2 is optional.
> The implementation in the patch is:
> I did not implement it exactly as defined in the Jira, instead:
> agent.sources.foo.selector.mapping.zebra = ch2
> agent.sources.foo.selector.optional.zebra = ch1
> Therefore the behavior of "mapping" keyword does not change, instead the optional keyword
simply adds more channels as optional channels. If "mapping" keyword does not exist for the
header or has no values, then the event is written to default channel and optional channels.
If optional is empty then even is simply written to channels specificied by "mapping"(if empty,
then to default channels). Basically mapping and optional means: write to channels specified
by "mapping" and if available, to the ones specified by "optional." If mapping is empty write
to default, and optionally to optional channels. 

This message was sent by Atlassian JIRA

View raw message