hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Sreekumar <hsreeku...@clickable.com>
Subject Re: Using ganglia to monitor hadoop
Date Tue, 23 Nov 2010 05:33:11 GMT
Hey Matthew,

          Got it working now.... had to restart the cluster. I am not sure
if restarting the cluster is what did the trick though. Even after
restarting, I had no metrics for about half an hour and then they started
coming. Thanks a lot for helping me set it up!

Cheers,
Hari

On Tue, Nov 23, 2010 at 1:40 AM, Matthew E. Kennedy <matt.kennedy@spadac.com
> wrote:

> I started to get mapred and jvm metrics without restarting the cluster, but
> it did require a restart to start getting dfs stats.  Yes, you should see
> the new metrics appear in the dropdown.
>
> On Nov 22, 2010, at 12:21 PM, Hari Sreekumar wrote:
>
> > Hi Matt,
> >       Tried using the specified(default) mcast join ip address. Still no
> > new metrics. Do I have to restart the hadoop cluster for the changes to
> take
> > effect? The new metrics will appear in the "Metrics" dropdown menu of
> > ganglia UI home page right? Or is there some other link for custom
> metrics?
> >
> > thanks,
> > Hari
> >
> > On Mon, Nov 22, 2010 at 9:08 PM, Matthew E. Kennedy <
> matt.kennedy@spadac.com
> >> wrote:
> >
> >> Hari,
> >>
> >> I have this set up and working and it looks to me like your *.servers
> >> properties are set incorrectly.  Point that property to the multicast
> >> address that is listed in your gmond.conf file on port 8649.
> >>
> >> You can find the multicast address with:
> >> cat /etc/gmond.conf | grep mcast
> >>
> >> an example of the config follows:
> >> dfs.servers=<mcast_join_ip_addr>:8649
> >>
> >> Good luck,
> >> Matt
> >>
> >> On Nov 22, 2010, at 8:20 AM, Hari Sreekumar wrote:
> >>
> >>> Hi Sagar,
> >>>
> >>>       I followed that article. But I am not clear about some things:
> >>> 1. Should I use port 8650 or 8649? I have tried both actually, and 8649
> >>> makes sense, but just to confirm. (I have set tcp_accept_channel as
> 8650)
> >>> 2. In hadoop-metrics.properties, should I mention all my nodes as I
> have
> >>> done or only the local node? e.g, localhost:8649 everywhere?
> >>>
> >>> I have configured gmetad.conf, and I am able to see metrics from all my
> >>> nodes in my ganglia UI. I just don't see any hadoop specific metrics.
> >>>
> >>> Thanks,
> >>> Hari
> >>>
> >>> On Mon, Nov 22, 2010 at 6:42 PM, Sagar Shukla <
> >> sagar_shukla@persistent.co.in
> >>>> wrote:
> >>>
> >>>> Hi Hari,
> >>>>  Not sure if you have already followed article -
> >>>> http://wiki.apache.org/hadoop/GangliaMetrics to configure Ganglia for
> >>>> Hadoop Monitoring. Also in the details given by you, I do not see any
> >>>> configuration settings done in gmetad config. You will need to define
> >> the
> >>>> hosts from whom metrics should be collected. Once this configuration,
> >> you
> >>>> should see the host in gmetad UI interface.
> >>>>
> >>>> Thanks,
> >>>> Sagar
> >>>>
> >>>> -----Original Message-----
> >>>> From: Hari Sreekumar [mailto:hsreekumar@clickable.com]
> >>>> Sent: Monday, November 22, 2010 6:25 PM
> >>>> To: common-user@hadoop.apache.org
> >>>> Subject: Using ganglia to monitor hadoop
> >>>>
> >>>> Hi,
> >>>>
> >>>>      I have set up ganglia for my cluster, and it works fine. What are
> >>>> the changes I need to make to make ganglia show hadoop related
> >> parameters?
> >>>> My gmond/gmetad config is default except for one change in gmond.conf
> >> for
> >>>> all nodes:
> >>>>
> >>>> /* You can specify as many tcp_accept_channels as you like to share
> >>>> an xml description of the state of the cluster */
> >>>> tcp_accept_channel {
> >>>> port = 8650
> >>>> }
> >>>>
> >>>> I have changed the hadoop-metrics.properties file as follows:
> >>>>
> >>>> # Configuration of the "dfs" context for ganglia
> >>>> dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> >>>> dfs.period=10
> >>>> dfs.servers=hadoop1:8649 hadoop2:8649 hadoop3:8649 ejabber:8649
> >>>>
> >>>>
> >>>> # Configuration of the "mapred" context for null
> >>>> #mapred.class=org.apache.hadoop.metrics.spi.NullContext
> >>>>
> >>>> # Configuration of the "mapred" context for file
> >>>> #mapred.class=org.apache.hadoop.metrics.file.FileContext
> >>>> #mapred.period=10
> >>>> #mapred.fileName=/tmp/mrmetrics.log
> >>>>
> >>>> # Configuration of the "mapred" context for ganglia
> >>>> mapred.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> >>>> mapred.period=10
> >>>> mapred.servers=hadoop1:8649 hadoop2:8649 hadoop3:8649 ejabber:8649
> >>>>
> >>>>
> >>>> # Configuration of the "jvm" context for null
> >>>> #jvm.class=org.apache.hadoop.metrics.spi.NullContext
> >>>>
> >>>> # Configuration of the "jvm" context for file
> >>>> #jvm.class=org.apache.hadoop.metrics.file.FileContext
> >>>> #jvm.period=10
> >>>> #jvm.fileName=/tmp/jvmmetrics.log
> >>>>
> >>>> # Configuration of the "jvm" context for ganglia
> >>>> jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> >>>> jvm.period=10
> >>>> jvm.servers=hadoop1:8649 hadoop2:8649 hadoop3:8649 ejabber:8649
> >>>>
> >>>> rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> >>>> rpc.period=10
> >>>> rpc.servers=hadoop1:8649 hadoop2:8649 hadoop3:8649 ejabber:8649
> >>>>
> >>>> hadoop1,2,3 and ejabber are my nodes.
> >>>>
> >>>> But I am unable to see any new metrics in ganglia's drop down menu.
Is
> >>>> there
> >>>> something else I need to do as well? I have restarted gmond on all
> >> nodes.
> >>>>
> >>>> thanks,
> >>>> Hari
> >>>>
> >>>> DISCLAIMER
> >>>> ==========
> >>>> This e-mail may contain privileged and confidential information which
> is
> >>>> the property of Persistent Systems Ltd. It is intended only for the
> use
> >> of
> >>>> the individual or entity to which it is addressed. If you are not the
> >>>> intended recipient, you are not authorized to read, retain, copy,
> print,
> >>>> distribute or use this message. If you have received this
> communication
> >> in
> >>>> error, please notify the sender and delete all copies of this message.
> >>>> Persistent Systems Ltd. does not accept any liability for virus
> infected
> >>>> mails.
> >>>>
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message