incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Graham <billgra...@gmail.com>
Subject Re: suggested changes to tools/init.d scripts
Date Tue, 12 Jan 2010 20:15:50 GMT
Thanks Eric, that makes sense. I'm going to use the Hadoop-style start-*.sh
scripts instead, since they seem to work better for my needs.

The agent and data processor start/stop scripts work for me, but the
bin/start-collectors.sh scripts fails:

$ sda bin/start-collectors.sh
sed: -e expression #1, char 11: unknown option to `s'

Has anyone else seen this?

It also seems like this script ultimately invokes slaves.sh, which looks for
hostnames in conf/agents. Is this a bug or are collectors expected to run on
the agents nodes? I thought they were more typically run on the Hadoop data
nodes.


On Tue, Jan 12, 2010 at 10:50 AM, Eric Yang <eyang@yahoo-inc.com> wrote:

> Hi Bill,
>
> The scripts are over due for a clean up.  Majority of the scripts were
> designed to work in a specific environment for Yahoo.
>
> On 1/12/10 10:24 AM, "Bill Graham" <billgraham@gmail.com> wrote:
>
> > Hi,
> >
> > To get the scripts in the tools/init.d folder to run in our environment I
> had
> > to make a few of the same tweaks to each of them. I'd like to suggest
> some
> > refactoring of these scripts to make them more easy to use. These are the
> > issues I came across:
> >
> > 1. The scripts try to write lock files to /var/lock/subsys, which is
> owned by
> > root. Can we change this location to be somewhere that doesn't require
> root
> > access?
>
> This was designed to run as /etc/init.d script.  I would recommend to
> remove
> tools/init.d and /etc/init.d script, and focus on using start-*.sh
> stop-*.sh
> commands like hadoop.
>
> > 2. The actual run command does an su to the CHUKWA_USER which also caused
> > problems for us. It seems like it would be cleaner to not embed the su
> calls
> > in the script, but instead allow the user to su when they run the script
> > (which worked much better for us). That way everything done by the script
> > would be done by the same user.
>
> Base on previous experience, we had a user who were running everything as
> root.  Su was put in there to safe guard the program from that user.  It
> was
> a pain to clean up hundred of machines with root owned files.  In a
> production environment, it is probably not Chukwa's responsibility to safe
> guard permission.  +1 on removing su.
>
> > 3. Each script has CHUKWA_HOME, CHUKWA_CONF_DIR and CHUKWA_USER hard
> coded.
> > Hard coded defaults is ok, but the ability to override them without
> modifying
> > the scripts would be ideal.
> >
>
> Current documentation doesn┬╣t talk about those properties are configurable
> during build time.  If you compile from trunk or 0.3 branch, you can copy
> default.properties to build.properties, and specify the path and running
> user.  However, the replace only applies to /etc/init.d/chukwa-* scripts,
> and install from RPM.  (Chukwa has an "ant rpm" target.)  tools/init.d was
> meant as a template, hence no replacement took place.
>
> > Let me know if you think these changes make sense and I'll open a JIRA.
>
> Yes, open a jira, and we can discuss in depth of the changes.  I would like
> to see /etc/init.d and tools/init.d to be removed completely, and use
> start-*.sh and stop-*.sh command to be consistent with hadoop.
>
> Regards,
> Eric
>
>

Mime
View raw message