hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wojciech Indyk <wojciechin...@gmail.com>
Subject Re: HBase monitoring
Date Tue, 07 Jul 2015 09:28:54 GMT
Yes, the explanation helped a lot. Thank you!

Kindly regards
Wojciech Indyk


2015-07-02 8:44 GMT+02:00 Esteban Gutierrez <esteban@cloudera.com>:
> Hi Wojciech,
>
> From the memory metrics and the up time of the RS reported by top seems
> that JVM has been running for a while, so a good part of the memory pages
> available for the JVM process have been touched and now are considered as
> used by to OS. Same thing can happen if -Xms is present and its larger than
> the default or equal to -Xmx causing those memory pages to be "touched" and
> considered by the OS as used even when the JVM hasn't claimed that memory
> for object creation.
>
> BTW, usually when you monitor memory utilization by HBase, you only monitor
> the JVM metrics and not the memory usage reported by the OS for the HBase
> process, since that usually is (or will become) a constant value during the
> life of the process.
>
> I hope that helps.
>
> esteban.
>
>
>
>
> --
> Cloudera, Inc.
>
>
> On Wed, Jun 24, 2015 at 2:27 AM, Wojciech Indyk <wojciechindyk@gmail.com>
> wrote:
>
>> hello!
>> I try to monitor memory of my regionservers. I have HBase 0.98 on CDH
>> 5.3.1.
>> I can see inconsistency between HBase metrics (here is metrics dump
>> from HBase UI):
>> "name" : "Hadoop:service=HBase,name=JvmMetrics",
>>     "modelerType" : "JvmMetrics",
>>     "tag.Context" : "jvm",
>>     "tag.ProcessName" : "IPC",
>>     "tag.SessionId" : "",
>>     "tag.Hostname" : "sqhadoop04.gazeta.pl",
>>     "MemNonHeapUsedM" : 93.26676,
>>     "MemNonHeapCommittedM" : 94.89844,
>>     "MemNonHeapMaxM" : -9.536743E-7,
>>     "MemHeapUsedM" : 8513.122,
>>     "MemHeapCommittedM" : 20330.25,
>>     "MemHeapMaxM" : 20330.25,
>>     "MemMaxM" : 20330.25,
>> and system "top" monitoring (all of my hbase processes on the
>> regionserver):
>> PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>> 19700 hbase     20   0 22.3g  16g  23m S  1.3  8.7 479:47.53 java
>> 19874 hbase     20   0  107m 2404  556 S  0.7  0.0   9:16.13 hbase.sh
>> 19873 hbase     20   0  105m  908  556 S  0.0  0.0   0:00.00 hbase.sh
>>
>> As we can see there is 16GB according to "top" and ~8.5GB according to
>> HBase. Why these values are different?
>>
>> Kindly regards
>> Wojciech Indyk
>>

Mime
View raw message