hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <...@apache.org>
Subject Re: Hadoop tool-kit for monitoring
Date Tue, 17 May 2011 22:16:58 GMT
Also, it seems like Ganglia would be very well complemented by Nagios
to allow you to monitor an overall health of your cluster.
--
  Take care,
Konstantin (Cos) Boudnik
2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622

Disclaimer: Opinions expressed in this email are those of the author,
and do not necessarily represent the views of any company the author
might be affiliated with at the moment of writing.

On Tue, May 17, 2011 at 15:15, Allen Wittenauer <aw@apache.org> wrote:
>
> On May 17, 2011, at 3:11 PM, Mark question wrote:
>
>> So what other memory consumption tools do you suggest? I don't want to do it
>> manually and dump statistics into file because IO will affect performance
>> too.
>
>        We watch memory with Ganglia.  We also tune our systems such that a task
will only take X amount.  In other words, given an 8gb RAM:
>
>        1gb for the OS
>        1gb for the TT and DN
>        6gb for all tasks
>
>        if we assume each task will take max 1gb, then we end up with 3 maps and 3
reducers.
>
>        Keep in mind that the mem consumed is more than just JVM heap size.

Mime
View raw message