hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Massie <m...@cloudera.com>
Subject Re: Monitoring hadoop?
Date Fri, 05 Jun 2009 16:33:24 GMT
Anthony-

The ganglia web site is at http://ganglia.info/ with documentation in  
a wiki at http://ganglia.wiki.sourceforge.net/.  There is also a good  
wiki page at IBM as well http://www.ibm.com/developerworks/wikis/display/WikiPtype/ganglia

  .  Ganglia packages are available for most distributions to help  
with installation so make sure to grep for ganglia with your favorite  
package manager (e.g. aptitude, yum, etc).  Ganglia will give you more  
information about your cluster than just Hadoop metrics.  You'll get  
CPU, load, memory, disk and network  monitoring as well for free.

You can see live demos of ganglia at http://ganglia.info/?page_id=69.

Good luck.

-Matt

On Jun 5, 2009, at 7:10 AM, Brian Bockelman wrote:

> Hey Anthony,
>
> Look into hooking your Hadoop system into Ganglia; this produces  
> about 20 real-time statistics per node.
>
> Hadoop also does JMX, which hooks into more "enterprise"-y  
> monitoring systems.
>
> Brian
>
> On Jun 5, 2009, at 8:55 AM, Anthony McCulley wrote:
>
>> Hey all,
>> I'm currently tasked to come up with a web/flex-based
>> visualization/monitoring system for a cloud system using hadoop as  
>> part of a
>> university research project.  I was wondering if I could elicit some
>> feedback from all of you with regards to:
>>
>>
>>  - If you were an engineer of a cloud system running hadoop, what
>>  information would you be interested in capturing, viewing,  
>> monitoring, etc?
>>  - Is there any sort of real-time stats or monitoring currently  
>> available
>>  for hadoop?  if so, is in a web-friendly format?
>>
>> Thanks in advance,
>>
>> - Anthony
>


Mime
View raw message