hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Washusen <...@reactive.org>
Subject Re: Optimizations for random read performance
Date Tue, 16 Feb 2010 00:16:16 GMT
Maybe the block cache is thrashing?

If you are regularly writing data to your tables then it's possible that the
block cache is no longer being effective.  On the region server web UI check
the blockCacheHitRatio value.  You want this value to be high (0 - 100).  If
this value is low it means that HBase has to go to disk to fetch blocks of
data.  You can control the amount of VM memory that HBase allocates to the
block cache using the "hfile.block.cache.size" property (default is 0.2
(20%)).

Cheers,
Dan

On 16 February 2010 10:45, James Baldassari <james@dataxu.com> wrote:

> Hi,
>
> Does anyone have any tips to share regarding optimization for random
> read performance?  For writes I've found that setting a large write
> buffer and setting auto-flush to false on the client side significantly
> improved put performance.  Are there any similar easy tweaks to improve
> random read performance?
>
> I'm using HBase 0.20.3 in a very read-heavy real-time system with 1
> master and 3 region servers.  It was working ok for a while, but today
> there was a severe degradation in read performance.  Restarting Hadoop
> and HBase didn't help, are there are no errors in the logs.  Read
> performance starts off around 1,000-2,000 gets/second but quickly
> (within minutes) drops to around 100 gets/second.
>
> I've already looked at the performance tuning wiki page.  On the server
> side I've increased hbase.regionserver.handler.count from 10 to 100, but
> it didn't help.  Maybe this is expected because I'm only using a single
> client to do reads.  I'm working on implementing a client pool now, but
> I'm wondering if there are any other settings on the server or client
> side that might improve things.
>
> Thanks,
> James
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message