hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthew E. Kennedy" <matt.kenn...@spadac.com>
Subject Re: Using ganglia to monitor hadoop
Date Mon, 22 Nov 2010 20:10:12 GMT
I started to get mapred and jvm metrics without restarting the cluster, but it did require
a restart to start getting dfs stats.  Yes, you should see the new metrics appear in the dropdown.

On Nov 22, 2010, at 12:21 PM, Hari Sreekumar wrote:

> Hi Matt,
>       Tried using the specified(default) mcast join ip address. Still no
> new metrics. Do I have to restart the hadoop cluster for the changes to take
> effect? The new metrics will appear in the "Metrics" dropdown menu of
> ganglia UI home page right? Or is there some other link for custom metrics?
> 
> thanks,
> Hari
> 
> On Mon, Nov 22, 2010 at 9:08 PM, Matthew E. Kennedy <matt.kennedy@spadac.com
>> wrote:
> 
>> Hari,
>> 
>> I have this set up and working and it looks to me like your *.servers
>> properties are set incorrectly.  Point that property to the multicast
>> address that is listed in your gmond.conf file on port 8649.
>> 
>> You can find the multicast address with:
>> cat /etc/gmond.conf | grep mcast
>> 
>> an example of the config follows:
>> dfs.servers=<mcast_join_ip_addr>:8649
>> 
>> Good luck,
>> Matt
>> 
>> On Nov 22, 2010, at 8:20 AM, Hari Sreekumar wrote:
>> 
>>> Hi Sagar,
>>> 
>>>       I followed that article. But I am not clear about some things:
>>> 1. Should I use port 8650 or 8649? I have tried both actually, and 8649
>>> makes sense, but just to confirm. (I have set tcp_accept_channel as 8650)
>>> 2. In hadoop-metrics.properties, should I mention all my nodes as I have
>>> done or only the local node? e.g, localhost:8649 everywhere?
>>> 
>>> I have configured gmetad.conf, and I am able to see metrics from all my
>>> nodes in my ganglia UI. I just don't see any hadoop specific metrics.
>>> 
>>> Thanks,
>>> Hari
>>> 
>>> On Mon, Nov 22, 2010 at 6:42 PM, Sagar Shukla <
>> sagar_shukla@persistent.co.in
>>>> wrote:
>>> 
>>>> Hi Hari,
>>>>  Not sure if you have already followed article -
>>>> http://wiki.apache.org/hadoop/GangliaMetrics to configure Ganglia for
>>>> Hadoop Monitoring. Also in the details given by you, I do not see any
>>>> configuration settings done in gmetad config. You will need to define
>> the
>>>> hosts from whom metrics should be collected. Once this configuration,
>> you
>>>> should see the host in gmetad UI interface.
>>>> 
>>>> Thanks,
>>>> Sagar
>>>> 
>>>> -----Original Message-----
>>>> From: Hari Sreekumar [mailto:hsreekumar@clickable.com]
>>>> Sent: Monday, November 22, 2010 6:25 PM
>>>> To: common-user@hadoop.apache.org
>>>> Subject: Using ganglia to monitor hadoop
>>>> 
>>>> Hi,
>>>> 
>>>>      I have set up ganglia for my cluster, and it works fine. What are
>>>> the changes I need to make to make ganglia show hadoop related
>> parameters?
>>>> My gmond/gmetad config is default except for one change in gmond.conf
>> for
>>>> all nodes:
>>>> 
>>>> /* You can specify as many tcp_accept_channels as you like to share
>>>> an xml description of the state of the cluster */
>>>> tcp_accept_channel {
>>>> port = 8650
>>>> }
>>>> 
>>>> I have changed the hadoop-metrics.properties file as follows:
>>>> 
>>>> # Configuration of the "dfs" context for ganglia
>>>> dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext
>>>> dfs.period=10
>>>> dfs.servers=hadoop1:8649 hadoop2:8649 hadoop3:8649 ejabber:8649
>>>> 
>>>> 
>>>> # Configuration of the "mapred" context for null
>>>> #mapred.class=org.apache.hadoop.metrics.spi.NullContext
>>>> 
>>>> # Configuration of the "mapred" context for file
>>>> #mapred.class=org.apache.hadoop.metrics.file.FileContext
>>>> #mapred.period=10
>>>> #mapred.fileName=/tmp/mrmetrics.log
>>>> 
>>>> # Configuration of the "mapred" context for ganglia
>>>> mapred.class=org.apache.hadoop.metrics.ganglia.GangliaContext
>>>> mapred.period=10
>>>> mapred.servers=hadoop1:8649 hadoop2:8649 hadoop3:8649 ejabber:8649
>>>> 
>>>> 
>>>> # Configuration of the "jvm" context for null
>>>> #jvm.class=org.apache.hadoop.metrics.spi.NullContext
>>>> 
>>>> # Configuration of the "jvm" context for file
>>>> #jvm.class=org.apache.hadoop.metrics.file.FileContext
>>>> #jvm.period=10
>>>> #jvm.fileName=/tmp/jvmmetrics.log
>>>> 
>>>> # Configuration of the "jvm" context for ganglia
>>>> jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext
>>>> jvm.period=10
>>>> jvm.servers=hadoop1:8649 hadoop2:8649 hadoop3:8649 ejabber:8649
>>>> 
>>>> rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext
>>>> rpc.period=10
>>>> rpc.servers=hadoop1:8649 hadoop2:8649 hadoop3:8649 ejabber:8649
>>>> 
>>>> hadoop1,2,3 and ejabber are my nodes.
>>>> 
>>>> But I am unable to see any new metrics in ganglia's drop down menu. Is
>>>> there
>>>> something else I need to do as well? I have restarted gmond on all
>> nodes.
>>>> 
>>>> thanks,
>>>> Hari
>>>> 
>>>> DISCLAIMER
>>>> ==========
>>>> This e-mail may contain privileged and confidential information which is
>>>> the property of Persistent Systems Ltd. It is intended only for the use
>> of
>>>> the individual or entity to which it is addressed. If you are not the
>>>> intended recipient, you are not authorized to read, retain, copy, print,
>>>> distribute or use this message. If you have received this communication
>> in
>>>> error, please notify the sender and delete all copies of this message.
>>>> Persistent Systems Ltd. does not accept any liability for virus infected
>>>> mails.
>>>> 
>> 
>> 


Mime
View raw message