hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gulfie <gul...@haruko.grotto-group.com>
Subject Re: Which metrics to track?
Date Tue, 11 Sep 2012 03:08:21 GMT

On Mon, Sep 10, 2012 at 08:16:29PM +0000, Jones, Robert wrote:
> Hello all, I am a sysadmin and do not know that much about Hadoop.  I run a stats/metrics
tracking system that logs stats over time so you can look at historical and current data and
perform some trend analysis.  I know I can access several hadoop metrics via jmx by going
to http://localhost:50070/jmx?qry=hadoop:* and I've got a script that parses all that data
so that I can stuff any of it that I want into our stats system.
> So, with all that preamble, I have one question.  Which metrics are worth tracking? 
There are a *lot* of metrics returned via the jmx query but I doubt all of them are critically
important.  Which metrics are important to track if we want to watch for any trends or spikes
in our hadoop cluster?

	All of them?  Often the funny ignored metrics are the ones that you'll want to look at only
after something else triggers and you start looking into the causes.  At least keep the gziped
raw files.  If need be you can feed the files into a hadoop job for later analysis. 


> Please provide some education to this noob.  Thanks!
> --
> Bob Jones
> Linux Systems/Network Engineer
> ME Cloud Computing
> Advanced Systems Development, Inc.
> 1 (434) 964-3156
> ****************************************************************************** This message
and any attachments are solely for the intended recipient and may contain confidential or
privileged information. If you are not the intended recipient, any disclosure, copying, use,
or distribution of the information included in this message and any attachments is prohibited.
If you have received this communication in error, please notify us by reply e-mail and immediately
and permanently delete this message and any attachments. Any views or opinions presented in
this email are solely those of the author and do not necessarily represent those of ASD. Employees
of ASD are expressly required not to make defamatory statements and not to infringe or authorize
any infringement of copyright or any other legal right by email communications. Any such communication
is contrary to company policy and outside the scope of the employment of the individual concerned.
The company will not accept any liability in respect. ******************************************************************************

View raw message