hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jason hadoop <jason.had...@gmail.com>
Subject Re: Monitoring with Ganglia
Date Tue, 17 Mar 2009 15:25:39 GMT
Make all of your hadoop-metrics properties use the standard IP address of
your master node.
Then add a straight udp receive block to the gmond.conf of your master node.
Then point your gmetad.conf at your master node.

There are complete details in forthcoming book, and with this in it, should
be available in alpha soon.

On Tue, Mar 17, 2009 at 8:23 AM, Tamir Kamara <tamirkamara@gmail.com> wrote:

> I sent my gmond.conf in my previous email... and the address is like carlos
> wrote.
>
> I'll change the hadoop-metrics file and check again.
> However, I would prefer to use a method I'm more familiar with - like
> unicast tcp communication. Do you know what I need to change in ganglia and
> / or hadoop to use it ?
>
> Thanks.
>
>
> On Tue, Mar 17, 2009 at 5:16 PM, Brian Bockelman <bbockelm@cse.unl.edu
> >wrote:
>
> >
> > On Mar 17, 2009, at 10:08 AM, Carlos Valiente wrote:
> >
> >  On Tue, Mar 17, 2009 at 14:56, Tamir Kamara <tamirkamara@gmail.com>
> >> wrote:
> >>
> >>> I don't know too much about multicast... and I'm using the default
> gmond
> >>> conf file.
> >>>
> >>
> >> The default multicast address seems to be 239.2.11.71, so that's the
> >> one for your hadoop-metrics.properties.
> >>
> >
> > Yup, try that - although I could tell better if I had Tamir's gmond.conf,
> > of course.
> >
> >
> >>
> >>  Wouldn't using the multicast address mean I'll need to specify a
> >>> different
> >>> address for each node so that the data won't get to all nodes running
> >>> gmond
> >>>
> >>
> >>
> > The design of Ganglia is such that all the data goes at all the nodes
> > running gmond.  If you don't like it, Ganglia 3.1 supports non-multicast
> TCP
> > channels.
> >
> > For reference, our 200 node cluster has about 250KB/s of background
> chatter
> > on idle nodes, which is probably Ganglia-related.  It's an incredibly
> small
> > perturbation on network traffic.
> >
> > Brian
> >
> >
> >  I'm not an expert, either --- I'm using the same multicast address on
> >> all nodes in my cluster. On each node, tcpdump shows incoming Ganglia
> >> traffic from every other node to the multicast address. It's usually a
> >> burst of about  200 UDP packets every 4 seconds or so (for a 6-node
> >> cluster), so the traffic overhead should be negligible.
> >>
> >> C
> >>
> >
> >
>



-- 
Alpha Chapters of my book on Hadoop are available
http://www.apress.com/book/view/9781430219422

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message