hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Baranau <alex.barano...@gmail.com>
Subject Re: memory usage too high in regionserver
Date Wed, 25 Aug 2010 17:59:46 GMT
Hello guys,

> HBase is a self memory tuning app

Is there any documentation on that or if you could point to some sources I
will be very thankful (even if sources are particular classes, please name
at least few to look at).

Thank you in advance,
Alex Baranau.

On Tue, Aug 10, 2010 at 1:25 AM, Ryan Rawson <ryanobjc@gmail.com> wrote:

> Hey,
>
> HBase is a self memory tuning app, unlike most Java apps you may have
> experience with.  We use as much memory as possible for memstore and
> block cache, but we also flush memory to keep overall usage below
> maximums.
>
> Enjoy,
> -ryan
>
> On Mon, Aug 9, 2010 at 3:22 PM, Jinsong Hu <jinsong_hu@hotmail.com> wrote:
> > you are right. now it shows up as
> >
> > request=180.5, regions=60, stores=61, storefiles=67,
> storefileIndexSize=143,
> > memstoreSize=20, compactionQueueSize=0, usedHeap=1417, maxHeap=6127,
> > blockCacheSize=1061944288, blockCacheFree=223059360,
> blockCacheCount=16239,
> > blockCacheHitRatio=14
> >
> > still 60 regions, and the usedHeap is only 1.4G. I already put all the
> > memory tunning suggested in the hbase-env.sh to do incremental GC.
> > looks the jvm still have drastic GC even with the incremental GC turned
> on.
> > now the GC log shows:
> >
> > 1462948.533: [GC 1462948.533: [ParNew: 138376K->1441K(153344K), 0.0037940
> > secs]
> > 1446591K->1309805K(6274432K) icms_dc=0 , 0.0038990 secs] [Times:
> user=0.03
> > sys=0
> > .00, real=0.00 secs]
> > 1463005.844: [GC 1463005.845: [ParNew: 137716K->17024K(153344K),
> 0.0139630
> > secs]
> > 1446081K->1327172K(6274432K) icms_dc=0 , 0.0140740 secs] [Times:
> user=0.11
> > sys=
> > 0.00, real=0.02 secs]
> >
> > which is reasonable.
> >
> > Jimmy.
> >
> > --------------------------------------------------
> > From: "Ryan Rawson" <ryanobjc@gmail.com>
> > Sent: Monday, August 09, 2010 3:07 PM
> > To: <user@hbase.apache.org>
> > Subject: Re: memory usage too high in regionserver
> >
> >> Unfortunately 'usedHeap' includes the unfound garbage...
> >>
> >> Perhaps you can post more of your GC log.
> >>
> >> On Mon, Aug 9, 2010 at 3:02 PM, Jinsong Hu <jinsong_hu@hotmail.com>
> wrote:
> >>>
> >>> Hi, There:
> >>>
> >>>  I am using cloudera cdh3 regionserver and today, I noticed that one of
> >>> the
> >>> regionserver memory usage is very high:
> >>>
> >>> request=8.8, regions=60, stores=61, storefiles=67,
> >>> storefileIndexSize=142,
> >>> memstoreSize=58, compactionQueueSize=0, usedHeap=5869, maxHeap=6127,
> >>> blockCacheSize=1061944288, blockCacheFree=223059360,
> >>> blockCacheCount=16239,
> >>> blockCacheHitRatio=14
> >>>
> >>> this shows that only 60 regions calused close to 6G of memory usage.
> This
> >>> is
> >>> very high.
> >>>
> >>> the gc-hbase.log shows:
> >>>
> >>> 1461575.642: [GC 1461575.643: [ParNew: 143538K->17024K(153344K),
> >>> 0.0487050
> >>> secs]
> >>> 5948479K->5846957K(6274432K) icms_dc=0 , 0.0488210 secs] [Times:
> >>> user=0.33
> >>> sys=
> >>> 0.00, real=0.05 secs]
> >>> 1461693.726: [GC 1461693.726: [ParNew: 153344K->17024K(153344K),
> >>> 0.0454440
> >>> secs]
> >>> 5983277K->5875331K(6274432K) icms_dc=0 , 0.0455680 secs] [Times:
> >>> user=0.31
> >>> sys=
> >>> 0.01, real=0.05 secs]
> >>>
> >>>
> >>> This shows that the regionserver is indeed using 6G of memory. very
> close
> >>> to
> >>> the setting of -Xmx6G limit.
> >>> and very soon the regionserver will be OOME.
> >>>
> >>>
> >>> if the above holds true, that means a regionserver can only support
> >>> around
> >>> 60 regions. I configured the region size to be default 256M.
> >>>
> >>> I checked the load on the server and the load is very light. cpu and
> disk
> >>> usage is very low. What can I do to drastically increase the number of
> >>> regions
> >>> supported per regionserver in hbase ?
> >>>
> >>>
> >>> Jimmy
> >>>
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message