hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Sreekumar <hsreeku...@clickable.com>
Subject Re: Using ganglia to monitor hadoop
Date Mon, 22 Nov 2010 17:21:39 GMT
Hi Matt,
       Tried using the specified(default) mcast join ip address. Still no
new metrics. Do I have to restart the hadoop cluster for the changes to take
effect? The new metrics will appear in the "Metrics" dropdown menu of
ganglia UI home page right? Or is there some other link for custom metrics?

thanks,
Hari

On Mon, Nov 22, 2010 at 9:08 PM, Matthew E. Kennedy <matt.kennedy@spadac.com
> wrote:

> Hari,
>
> I have this set up and working and it looks to me like your *.servers
> properties are set incorrectly.  Point that property to the multicast
> address that is listed in your gmond.conf file on port 8649.
>
> You can find the multicast address with:
> cat /etc/gmond.conf | grep mcast
>
> an example of the config follows:
> dfs.servers=<mcast_join_ip_addr>:8649
>
> Good luck,
> Matt
>
> On Nov 22, 2010, at 8:20 AM, Hari Sreekumar wrote:
>
> > Hi Sagar,
> >
> >        I followed that article. But I am not clear about some things:
> > 1. Should I use port 8650 or 8649? I have tried both actually, and 8649
> > makes sense, but just to confirm. (I have set tcp_accept_channel as 8650)
> > 2. In hadoop-metrics.properties, should I mention all my nodes as I have
> > done or only the local node? e.g, localhost:8649 everywhere?
> >
> > I have configured gmetad.conf, and I am able to see metrics from all my
> > nodes in my ganglia UI. I just don't see any hadoop specific metrics.
> >
> > Thanks,
> > Hari
> >
> > On Mon, Nov 22, 2010 at 6:42 PM, Sagar Shukla <
> sagar_shukla@persistent.co.in
> >> wrote:
> >
> >> Hi Hari,
> >>   Not sure if you have already followed article -
> >> http://wiki.apache.org/hadoop/GangliaMetrics to configure Ganglia for
> >> Hadoop Monitoring. Also in the details given by you, I do not see any
> >> configuration settings done in gmetad config. You will need to define
> the
> >> hosts from whom metrics should be collected. Once this configuration,
> you
> >> should see the host in gmetad UI interface.
> >>
> >> Thanks,
> >> Sagar
> >>
> >> -----Original Message-----
> >> From: Hari Sreekumar [mailto:hsreekumar@clickable.com]
> >> Sent: Monday, November 22, 2010 6:25 PM
> >> To: common-user@hadoop.apache.org
> >> Subject: Using ganglia to monitor hadoop
> >>
> >> Hi,
> >>
> >>       I have set up ganglia for my cluster, and it works fine. What are
> >> the changes I need to make to make ganglia show hadoop related
> parameters?
> >> My gmond/gmetad config is default except for one change in gmond.conf
> for
> >> all nodes:
> >>
> >> /* You can specify as many tcp_accept_channels as you like to share
> >>  an xml description of the state of the cluster */
> >> tcp_accept_channel {
> >> port = 8650
> >> }
> >>
> >> I have changed the hadoop-metrics.properties file as follows:
> >>
> >> # Configuration of the "dfs" context for ganglia
> >> dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> >> dfs.period=10
> >> dfs.servers=hadoop1:8649 hadoop2:8649 hadoop3:8649 ejabber:8649
> >>
> >>
> >> # Configuration of the "mapred" context for null
> >> #mapred.class=org.apache.hadoop.metrics.spi.NullContext
> >>
> >> # Configuration of the "mapred" context for file
> >> #mapred.class=org.apache.hadoop.metrics.file.FileContext
> >> #mapred.period=10
> >> #mapred.fileName=/tmp/mrmetrics.log
> >>
> >> # Configuration of the "mapred" context for ganglia
> >> mapred.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> >> mapred.period=10
> >> mapred.servers=hadoop1:8649 hadoop2:8649 hadoop3:8649 ejabber:8649
> >>
> >>
> >> # Configuration of the "jvm" context for null
> >> #jvm.class=org.apache.hadoop.metrics.spi.NullContext
> >>
> >> # Configuration of the "jvm" context for file
> >> #jvm.class=org.apache.hadoop.metrics.file.FileContext
> >> #jvm.period=10
> >> #jvm.fileName=/tmp/jvmmetrics.log
> >>
> >> # Configuration of the "jvm" context for ganglia
> >> jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> >> jvm.period=10
> >> jvm.servers=hadoop1:8649 hadoop2:8649 hadoop3:8649 ejabber:8649
> >>
> >> rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> >> rpc.period=10
> >> rpc.servers=hadoop1:8649 hadoop2:8649 hadoop3:8649 ejabber:8649
> >>
> >> hadoop1,2,3 and ejabber are my nodes.
> >>
> >> But I am unable to see any new metrics in ganglia's drop down menu. Is
> >> there
> >> something else I need to do as well? I have restarted gmond on all
> nodes.
> >>
> >> thanks,
> >> Hari
> >>
> >> DISCLAIMER
> >> ==========
> >> This e-mail may contain privileged and confidential information which is
> >> the property of Persistent Systems Ltd. It is intended only for the use
> of
> >> the individual or entity to which it is addressed. If you are not the
> >> intended recipient, you are not authorized to read, retain, copy, print,
> >> distribute or use this message. If you have received this communication
> in
> >> error, please notify the sender and delete all copies of this message.
> >> Persistent Systems Ltd. does not accept any liability for virus infected
> >> mails.
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message