incubator-bigtop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Will McQueen (JIRA)" <>
Subject [jira] [Commented] (BIGTOP-566) Flume NG pkg init script should allow user to customize the location of the conf dir
Date Fri, 27 Apr 2012 23:34:50 GMT


Will McQueen commented on BIGTOP-566:

Hi Roman,

I saw your comment after I already posted the patch. For #1, I agree. For #2:

>>with long running agents on each node having a configuration that changes only when
the agent gets restarted
The configuration can change at any time. There's a poller thread that checks for config changes
to the file every 30 secs.

I'm not sure what the right answer is to your other questions. For launching Flume NG, I was
comparing the level of customizability that Apache Flume NG offers to the way that Bigtop
packaging uses the init script to call flume-ng. But maybe that's the wrong comparison. So
let me back up.

The init script currently allows FLUME_AGENT_NAME and FLUME_CONF_FILE to be taken from the
environment so that the user can customize it. But maybe init scripts should not be customizable?
Maybe the init script should enforce a non-overridable FLUME_AGENT_NAME ("agent") and FLUME_CONF_FILE
("/etc/flume-ng/conf/flume.conf")? Maybe init scripts in general should enforce a particular
common scenario, like supporting only a single Flume agent to run on the host and with specific
options, and any deviation from that scenario should be handled by directly running the target
binary ("/usr/bin/flume-ng" in this case) with custom options for things like --conf, --conf-file,
and --name? Or, maybe if the init script is customizable, then it could use shell fragments
similar to what /usr/libexec/bigtop-detect-javahome does when it calls /etc/default/bigtop-utils?

I'd appreciate any light you could shed on standard conventions in this regard. Thank you.

> Flume NG pkg init script should allow user to customize the location of the conf dir
> ------------------------------------------------------------------------------------
>                 Key: BIGTOP-566
>                 URL:
>             Project: Bigtop
>          Issue Type: Bug
>          Components: General
>    Affects Versions: 0.4.0
>         Environment: all
>            Reporter: Will McQueen
>            Assignee: Bruno Mahé
>             Fix For: 0.4.0
>         Attachments: BIGTOP-566.patch, BIGTOP-566.patch
> A typical Flume NG invocation is:
>      bin/flume-ng --conf-file conf/  --conf conf --name
> ...and the corresponding invocation after installing the Flume NG bigtop package would
>      /usr/lib/flume-ng/bin/flume-ng --conf-file /etc/flume-ng/conf/flume.conf --conf
/etc/flume-ng/conf --name agent
> The Flume NG init script, installed as part of Bigtop Flume NG, allows us to modify:
> 1) the --conf-file value (via FLUME_CONF_FILE)
> 2) the --name value (via FLUME_AGENT_NAME)
> ...but not the --conf value. So the user loses some flexibility when using Bigtop Flume
NG over Apache Flume NG.
> I recommend that the /etc/init.d/flume-ng-agent script be modified to allow the user
to set FLUME_CONF_DIR. That is, in this init script, please change this:
>      FLUME_CONF_DIR=/etc/flume-ng/conf
> this:
>      FLUME_CONF_DIR=${FLUME_CONF_DIR:-/etc/flume-ng/conf}

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message