hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthew E. Kennedy" <matt.kenn...@spadac.com>
Subject Re: Using ganglia to monitor hadoop
Date Mon, 22 Nov 2010 15:38:10 GMT
Hari,

I have this set up and working and it looks to me like your *.servers properties are set incorrectly.
 Point that property to the multicast address that is listed in your gmond.conf file on port
8649.

You can find the multicast address with:
cat /etc/gmond.conf | grep mcast

an example of the config follows:
dfs.servers=<mcast_join_ip_addr>:8649

Good luck,
Matt

On Nov 22, 2010, at 8:20 AM, Hari Sreekumar wrote:

> Hi Sagar,
> 
>        I followed that article. But I am not clear about some things:
> 1. Should I use port 8650 or 8649? I have tried both actually, and 8649
> makes sense, but just to confirm. (I have set tcp_accept_channel as 8650)
> 2. In hadoop-metrics.properties, should I mention all my nodes as I have
> done or only the local node? e.g, localhost:8649 everywhere?
> 
> I have configured gmetad.conf, and I am able to see metrics from all my
> nodes in my ganglia UI. I just don't see any hadoop specific metrics.
> 
> Thanks,
> Hari
> 
> On Mon, Nov 22, 2010 at 6:42 PM, Sagar Shukla <sagar_shukla@persistent.co.in
>> wrote:
> 
>> Hi Hari,
>>   Not sure if you have already followed article -
>> http://wiki.apache.org/hadoop/GangliaMetrics to configure Ganglia for
>> Hadoop Monitoring. Also in the details given by you, I do not see any
>> configuration settings done in gmetad config. You will need to define the
>> hosts from whom metrics should be collected. Once this configuration, you
>> should see the host in gmetad UI interface.
>> 
>> Thanks,
>> Sagar
>> 
>> -----Original Message-----
>> From: Hari Sreekumar [mailto:hsreekumar@clickable.com]
>> Sent: Monday, November 22, 2010 6:25 PM
>> To: common-user@hadoop.apache.org
>> Subject: Using ganglia to monitor hadoop
>> 
>> Hi,
>> 
>>       I have set up ganglia for my cluster, and it works fine. What are
>> the changes I need to make to make ganglia show hadoop related parameters?
>> My gmond/gmetad config is default except for one change in gmond.conf for
>> all nodes:
>> 
>> /* You can specify as many tcp_accept_channels as you like to share
>>  an xml description of the state of the cluster */
>> tcp_accept_channel {
>> port = 8650
>> }
>> 
>> I have changed the hadoop-metrics.properties file as follows:
>> 
>> # Configuration of the "dfs" context for ganglia
>> dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext
>> dfs.period=10
>> dfs.servers=hadoop1:8649 hadoop2:8649 hadoop3:8649 ejabber:8649
>> 
>> 
>> # Configuration of the "mapred" context for null
>> #mapred.class=org.apache.hadoop.metrics.spi.NullContext
>> 
>> # Configuration of the "mapred" context for file
>> #mapred.class=org.apache.hadoop.metrics.file.FileContext
>> #mapred.period=10
>> #mapred.fileName=/tmp/mrmetrics.log
>> 
>> # Configuration of the "mapred" context for ganglia
>> mapred.class=org.apache.hadoop.metrics.ganglia.GangliaContext
>> mapred.period=10
>> mapred.servers=hadoop1:8649 hadoop2:8649 hadoop3:8649 ejabber:8649
>> 
>> 
>> # Configuration of the "jvm" context for null
>> #jvm.class=org.apache.hadoop.metrics.spi.NullContext
>> 
>> # Configuration of the "jvm" context for file
>> #jvm.class=org.apache.hadoop.metrics.file.FileContext
>> #jvm.period=10
>> #jvm.fileName=/tmp/jvmmetrics.log
>> 
>> # Configuration of the "jvm" context for ganglia
>> jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext
>> jvm.period=10
>> jvm.servers=hadoop1:8649 hadoop2:8649 hadoop3:8649 ejabber:8649
>> 
>> rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext
>> rpc.period=10
>> rpc.servers=hadoop1:8649 hadoop2:8649 hadoop3:8649 ejabber:8649
>> 
>> hadoop1,2,3 and ejabber are my nodes.
>> 
>> But I am unable to see any new metrics in ganglia's drop down menu. Is
>> there
>> something else I need to do as well? I have restarted gmond on all nodes.
>> 
>> thanks,
>> Hari
>> 
>> DISCLAIMER
>> ==========
>> This e-mail may contain privileged and confidential information which is
>> the property of Persistent Systems Ltd. It is intended only for the use of
>> the individual or entity to which it is addressed. If you are not the
>> intended recipient, you are not authorized to read, retain, copy, print,
>> distribute or use this message. If you have received this communication in
>> error, please notify the sender and delete all copies of this message.
>> Persistent Systems Ltd. does not accept any liability for virus infected
>> mails.
>> 


Mime
View raw message