incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ariel Rabkin <asrab...@gmail.com>
Subject Re: New to chukwa - problems running agent
Date Tue, 10 Nov 2009 18:06:40 GMT
OK.

So.  First, I sat down and did some extensive revisions to the admin
guide last night -- have a look at
http://hadoop.apache.org/chukwa/docs/r0.3.0/admin.html if you haven't
already.

Second.  If you want to start a single agent, interactively, you
should try something like:
$  bin/agent.sh local

You should see a log message on the screen telling you the agent started OK.

Then telnet to 9093 [or whatever other agent port you picked] and say:
add filetailer.FileTailingAdaptor Test /path/to/file 0

The contents of the file should appear on the screen attached to the agent.

If you ditch the local, and start the agent as a daemon and again add
an adaptor, it'll pick a collector from conf/collectors to talk to and
start tailing.   You should then look at the collector status page --
http://collector:portno/chukwa?ping=true and you should see that some
chunks have been received.

Congratulations!  You're now tailing a file.

conf/agents is optional; it's only used if you use the start-agents
script to invoke daemons in parallel via ssh.
conf/collectors is mandatory. It's the list of collectors to send data to.
conf/initial_adaptors is optional; it's the list of adapters to load
on startup by default.

On Tue, Nov 10, 2009 at 9:56 AM, Bill Graham <billgraham@gmail.com> wrote:
> Hi,
>
> I'm just getting set up with Chukwa and after a few bumps I was able to get
> my collector and data-processor running, but I'm now stuck on the agent. The
> docs on the admin guide seem a bit out of date, but I've followed the parts
> that seem relevant but still no dice.
>
> I configured conf/chukwa-agent-conf.xml, but I can't start the agent. When I
> try to I get this:
>
> [grahamb:/path/to/chukwa/chukwa-0.3.0]$ tools/init.d/chukwa-agent start
> Starting chukwa-agent: Usage: chukwa-daemon.sh [--config <conf-dir>]
> [--hosts hostlistfile] (start|stop) <chukwa-command> <args...>
>
> Looking under the covers a bit, that script is trying to invoke
> chukwa-daemon.sh like this:
> /path/to/chukwa/chukwa-0.3.0/bin/chukwa-daemon.sh
> --config=/path/to/chukwa/chukwa-0.3.0/conf --watchdog start agent.sh
> Usage: chukwa-daemon.sh [--config <conf-dir>] [--hosts hostlistfile]
> (start|stop) <chukwa-command> <args...>
>
> I tried that command with a --hosts file but still no luck. Also, are
> conf/collectors, conf/agents and conf/initial_adaptors still in use? The
> admin guide mentions them, but they're not in the distro.
>
> My understanding of how the agent is configured is still limited, so if
> anyone could explain (or show an example) of how to properly run it, that
> would be greatly appreciated. I'm trying to set up a basic Chukwa hello
> world that does the following:
>
> - I start an agent/adapter that tails a test log file
> - I can manually pipe log entries to the file from the command line with a
> timestamp and url like so:
>    http://domain.com?a=1&b=2
> - I can then write a M/R job that shows counts of a's and b's over time.
>
> If it seems like a hello world like this would be useful to others, I'd
> gladly contribute the documentation back. I just need a little help getting
> it implemented first...
>
> thanks,
> Bill
>



-- 
Ari Rabkin asrabkin@gmail.com
UC Berkeley Computer Science Department

Mime
View raw message