hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Trivial Update of "Chukwa Quick Start" by andyk
Date Mon, 17 Nov 2008 07:30:00 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The following page has been changed by andyk:

  == Compiling and installing Chukwa ==
   1. If Chukwa is in the hadoop contrib directory, you should be able to just say ''ant''
in the project root directory.
+  1. If you are building Chukwa in standalone mode, you should set the HADOOP_HOME environment
variable to point to your HADOOP installation (e.g. > export HADOOP_HOME=/path/to/hadoop).
This will ensure that Chukwa is built with the HDFS correct protocol version to be able to
talk to your running Hadoop Distributed File System (HDFS).
  == Configuring and starting the Collector ==
@@ -50, +51 @@

  == Running Chukwa on a Cluster ==
  The cluster deployment process is still under active development, thus it is possible that
the following instructions may not work yet, but they will soon, so please don't delete them.
Eventually, even the single machine setup (for newcomers to Chukwa who want to try it out
of the box on their) above will be replaced by the below process, renaming the conf/slaves.template
and conf/collectors.template files (to remove the .template suffix) for the defaults of localhost
for the collector and agent.
+ '''Configure Chukwa'''
+ 1. Specify your JAVA_HOME and HADOOP_HOME in conf/chukwa-env.sh
  1. Specify which hosts to run collectors on in the conf/collectors file.
- 1. Start the collectors in your cluster with the command <code>bin/start-collectors.sh</code>
  1. Like in Hadoop, you need to specify a set of nodes on which you want to run Chukwa agents
(similar to conf/slaves in Hadoop) using a conf/slaves file. The local agents on each machine
will also reference the conf/collectors file, selecting a collector at random from this list
to talk to. Thus, like Hadoop, it is common to run Chukwa from a shared file system where
all of the agents (i.e. slaves) can access the same conf files.
- 1. Start the agents by running <code>bin/start-agents.sh</code>
+ 1. Setup the initial adaptors you want to run on every agent in the chukwa cluster by copying
conf/initial_adaptors.template to conf/initial_adaptors and adding whichever adaptors you
see fit. See the ["Chukwa_Adaptors_List"] for a catalog of pre-made adaptors.
+ '''Run Chukwa'''
+ 1. run bin/start-all.sh to start all agents, collectors, data loaders, and schedule the
demux MapReduce job to run every 5 minutes over top of the datasink. Use bin/stop-all.sh to
shut everything down.
+ '''OR''' run the collectors and agents independently (without other scripts running as part
of startup and shutdown)
+ 1. Start the collectors in your cluster with the command <code>bin/start-collectors.sh</code>
(use bin/stop-collectors.sh to shut them down).
+ 1. Start the agents by running <code>bin/start-agents.sh</code> (use bin/stop-agents.sh
to shut them down).

View raw message