hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stack <st...@duboce.net>
Subject Re: HBase looses regions.
Date Fri, 22 May 2009 16:13:41 GMT
On Fri, May 22, 2009 at 7:21 AM, Kirill Shabunov <e2k_1@yahoo.com> wrote:

> Thanks a lot!
>
> I've tried, it seems to be much more stable overall. No component have
> crashed so far.


I'm glad.   If the RC is working for you, please vote for it over in
hbase-dev mailing list.


>
>
> I see some new messages, like this:
>  WARN org.apache.hadoop.hbase.regionserver.HLog:
> regionserver/0:0:0:0:0:0:0:0:60020.logFlusher took 1661ms optional sync to
> HLog; editcount=0
>

This is a new log I added.  It triggers every time an append or a "sync" on
the commit log takes > 1 second.  I added the log to highlight hdfs slow
periods taking on writes.  Its looking like we are going to have to put up a
pool of commit log writers if we want to keep up write rates.  The logging
is just so we begin to gather stats.

In the above, it looks like it happened just after file open (editcount=0).


I also changed some settings:
>
> 1. upped hbase.master.lease.period to allow regions to be busy longer.
>


Thats probably fine.



>
> 2. decreased some thresholds, where I hope it should reduce RAM usage:
>    - hbase.hregion.memcache.flush.size -- smaller buffers (?).
>    - hbase.hregion.max.filesize -- smaller the region, less RAM for the
> index (?).
>


Regards memory use, I would have left the above settings at default.  In
0.19.x era hbase, its the size of your storefile indices that takes up bulk
of heap.  See http://wiki.apache.org/hadoop/Hbase/FAQ#4.  This should all be
much better in 0.20.0 hbase.

St.Ack

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message