hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sunweiwei" <su...@asiainfo-linkage.com>
Subject 答复: 答复: 答复: 答复: what in regionserver's Used Heap
Date Tue, 27 May 2014 01:20:34 GMT
This is hadoop-metrics2-hbase.properties.

hbase.extendedperiod = 3600

# Configuration of the "hbase" context for ganglia
# Pick one: Ganglia 3.0 (former) or Ganglia 3.1 (latter)
# hbase.class=org.apache.hadoop.metrics.ganglia.GangliaContext
hbase.class=org.apache.hadoop.metrics.ganglia.GangliaContext31
hbase.period=10
hbase.servers=hadoop01:8663

# Configuration of the "jvm" context for ganglia
# Pick one: Ganglia 3.0 (former) or Ganglia 3.1 (latter)
# jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext
jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext31
jvm.period=10
jvm.servers=hadoop01:8663

# Configuration of the "rpc" context for ganglia
# Pick one: Ganglia 3.0 (former) or Ganglia 3.1 (latter)
# rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext
rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext31
rpc.period=10
rpc.servers=hadoop01:8663

#Ganglia following hadoop example
hbase.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
hbase.sink.ganglia.period=10

# default for supportsparse is false
*.sink.ganglia.supportsparse=true

.sink.ganglia.slope=jvm.metrics.gcCount=zero,jvm.metrics.memHeapUsedM=both
.sink.ganglia.dmax=jvm.metrics.threadsBlocked=70,jvm.metrics.memHeapUsedM=40

hbase.sink.ganglia.servers=hadoop01:8663

-----邮件原件-----
发件人: Ted Yu [mailto:yuzhihong@gmail.com] 
发送时间: 2014年5月26日 22:57
收件人: user@hbase.apache.org
主题: Re: 答复: 答复: 答复: what in regionserver's Used Heap

Can you pastebin the contents of conf/hadoop-metrics2-hbase.properties ?

Thanks


On Sun, May 25, 2014 at 11:37 PM, sunweiwei <sunww@asiainfo-linkage.com>wrote:

> Yes, I'm using metrics2 with ganglia.
> And I have closed ganglia, because there is too many region metrics.
> But  I don't delete metrics2 configuration in
> $HBASE_HOME/conf/hadoop-metrics2-hbase.properties.  Is this OK?
>
>
> Thanks
>
> -----邮件原件-----
> 发件人: Ted Yu [mailto:yuzhihong@gmail.com]
> 发送时间: 2014年5月26日 13:10
> 收件人: user@hbase.apache.org
> 主题: Re: 答复: 答复: what in regionserver's Used Heap
>
> Some metrics2 classes show up in the histogram.
>
> I am digging into these classes and polling developers who are more
> familiar with metrics2 system.
>
> FYI
>
>
> On Sun, May 25, 2014 at 6:42 PM, sunweiwei <sunww@asiainfo-linkage.com
> >wrote:
>
> > Hi
> >
> > This is jmap -histo result.
> > num     #instances         #bytes  class name
> > ----------------------------------------------
> >    1:      14917882     3396492464  [C
> >    2:       1996994     2118021808  [B
> >
> >  num     #instances         #bytes  class name
> > ----------------------------------------------
> >    1:      14917882     3396492464  [C
> >    2:       1996994     2118021808  [B
> >    3:      43341650     1733666000  java.util.LinkedHashMap$Entry
> >    4:      14453983     1156550896  [Ljava.util.HashMap$Entry;
> >    5:      14446577      924580928
> >  org.apache.hadoop.metrics2.lib.Interns$CacheWith2Keys$2
> >    6:      14816939      474142048  java.lang.String
> >    7:      14463510      347124240
> >  org.apache.hadoop.metrics2.lib.MetricsInfoImpl
> >    8:       6380066      255202640  org.apache.hadoop.hbase.KeyValue
> >    9:       7274264      174582336
> >  java.util.concurrent.ConcurrentSkipListMap$Node
> >   10:       3412039       81888936
> >  java.util.concurrent.ConcurrentSkipListMap$Index
> >   11:          9208       28516704  [I
> >   12:        259293       26673528  [Ljava.lang.Object;
> >   13:        424072       20355456  java.nio.HeapByteBuffer
> >   14:        418655       20095440  java.nio.HeapCharBuffer
> >   15:        657703       15784872
> >  org.apache.hadoop.metrics2.impl.MetricGaugeDouble
> >   16:        589926       14158224
> >  org.apache.hadoop.metrics2.impl.MetricCounterLong
> >   17:        522366       12536784
> >  org.apache.hadoop.metrics2.impl.MetricGaugeLong
> >   18:        504238       12101712  java.lang.Double
> >   19:        436872       10484928  com.google.protobuf.LiteralByteString
> >   20:         72414        9990968  <constMethodKlass>
> >   21:         72414        9857616  <methodKlass>
> >   22:         55752        9354400
> >  [Ljava.util.concurrent.ConcurrentHashMap$HashEntry;
> >   23:        103154        7427088
> >  org.apache.hadoop.hbase.protobuf.generated.ClientProtos$MutationProto
> >   24:        145924        7004352
> >  org.apache.hadoop.hbase.monitoring.MonitoredTaskImpl
> >   25:          5322        6835120  <constantPoolKlass>
> >   26:        251260        6030240  java.util.ArrayList
> >   27:         94355        5979040  <symbolKlass>
> >   28:        103170        5777520
> >
>  org.apache.hadoop.hbase.protobuf.generated.ClientProtos$MutationProto$ColumnValue$QualifierValue
> >   29:        238244        5717856
> >  java.util.Collections$UnmodifiableRandomAccessList
> >   30:        130150        5206000  java.util.TreeMap$Entry
> >   31:        103160        4951680
> >
>  org.apache.hadoop.hbase.protobuf.generated.ClientProtos$MutationProto$ColumnValue
> >
> >
> > Thanks
> > -----邮件原件-----
> > 发件人: Ted Yu [mailto:yuzhihong@gmail.com]
> > 发送时间: 2014年5月23日 22:34
> > 收件人: user@hbase.apache.org
> > 主题: Re: 答复: what in regionserver's Used Heap
> >
> > You can use jmap with -histo option to find out heap consumption:
> > http://docs.oracle.com/javase/7/docs/technotes/tools/share/jmap.html
> >
> > Please pastebin jmap output.
> >
> > Cheers
> >
> >
> > On Thu, May 22, 2014 at 11:30 PM, sunweiwei <sunww@asiainfo-linkage.com
> > >wrote:
> >
> > > Hi Ted
> > >
> > > I have checked regionsever:60030/rs-status#regionMemstoreStats.
> Memstore
> > > and Cache Size is very small.
> > > But Used Heap is very large.
> > > Is there Any other thing in the Used Heap, and use lots of heaps?
> > >
> > > Thanks
> > >
> > > -----邮件原件-----
> > > 发件人: Ted Yu [mailto:yuzhihong@gmail.com]
> > > 发送时间: 2014年5月23日 11:59
> > > 收件人: user@hbase.apache.org
> > > 主题: Re: what in regionserver's Used Heap
> > >
> > > On regionsever:60030/rs-status#regionMemstoreStats, you should be able
> to
> > > see how much memstore each region uses.
> > >
> > > You can also use ganglia, etc to view the metrics.
> > > See http://hbase.apache.org/book.html#hbase_metrics (15.4.4.3 and
> > > 15.4.4.7)
> > >
> > > Cheers
> > >
> > >
> > > On Thu, May 22, 2014 at 8:39 PM, sunweiwei <sunww@asiainfo-linkage.com
> > > >wrote:
> > >
> > > > Hi
> > > >
> > > >
> > > >
> > > > I'm using hbase0.96.
> > > >
> > > >
> > > >
> > > > In hbase web, I see :
> > > >
> > > > Max Heap is 16.0 G
> > > >
> > > > Memstore Size is 208.6 M
> > > >
> > > > Cache Size is  138M
> > > >
> > > > Cache Free is 6.2G
> > > >
> > > >
> > > >
> > > > And Used Heap is 10.2 G.
> > > >
> > > >
> > > >
> > > > Used Heap is  large.  I want to know  what   could be  in  then Used
> > > Heap.
> > > >
> > > >
> > > >
> > > >
> > > > Thanks.
> > > >
> > > >
> > >
> > >
> >
> >
>
>


Mime
View raw message