hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chandraprakash Bhagtani <cpbhagt...@gmail.com>
Subject Re: Performance tools for Hadoop
Date Wed, 03 Mar 2010 13:04:06 GMT
Hi Devender,

Currently there are 2 ways to analyze performance of hadoop cluster and jobs

 1. Hadoop Vaidya: is a performance diagnostic tool for hadoop jobs which
executes a set of rules against the job counters and gives a report of
performance improvement areas as a result. But Hadoop Vaidya is a separate
contrib module and has a limited set of rules as of now.

 2. Hadoop Metrics(used with Ganglia or Nagios): All  Hadoop daemons expose
runtime metrics which are then analyzed by some cluster monitoring system
like Ganglia or Nagios. Hadoop Metrics has a lot of dependencies on third
party libraries and limited number of metrics as of now.

We are enhancing Hadoop job analysis web UI which will include some features
of Vaidya and Metrics as a feature in Hadoop itself.
It will have no dependencies on any third party libraries. You will have
complete analytic and diagnostic report of your job by just a single click.

We are planning to contribute a patch soon.

On Wed, Mar 3, 2010 at 5:57 PM, Devender <dkumar@qasource.com> wrote:

> Hi Team,
> Is there anyone who can suggest any performance tools that can be used on
> the hadoop environment?
> Regards,
> Dev

Thanks & Regards,
Chandra Prakash Bhagtani,
Impetus Infotech (india) Pvt Ltd.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message