hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gaojinchao <gaojinc...@huawei.com>
Subject Re: Hmaster crashes caused by splitting log.
Date Thu, 23 Jun 2011 03:40:02 GMT
Thanks, You are right.  :) 

I will test it.


-----邮件原件-----
发件人: Ted Yu [mailto:yuzhihong@gmail.com] 
发送时间: 2011年6月23日 11:00
收件人: dev@hbase.apache.org
主题: Re: Hmaster crashes caused by splitting log.

I guess you have seen this call directly below the following code snippet:
      checkForErrors();
I think checking thrown should be kept in the condition.

I assume we can lift the above call before dataAvailable.notifyAll()

My 2 cents.

On Wed, Jun 22, 2011 at 7:00 PM, Gaojinchao <gaojinchao@huawei.com> wrote:

>
> Because Master usually<javascript:void(0)> uses little memory. So its
> memory is 4G.
>
> DFS block is 256M and hbase.regionserver.maxlogs is 32. One region server
> can save max 8G Hlog.
>
> In my performance cluster(0.90.3), The Hmaster memory from 100 M up to 4G
> when one region server crashed.
>
> I dug it and found the flow control does not work when write thread is
> normal.
>
>
> // If we crossed the chunk threshold, wait for more space to be available
>      synchronized (dataAvailable) {
>        while (totalBuffered > maxHeapUsage && thrown == null) {
>          LOG.debug("Used " + totalBuffered + " bytes of buffered edits,
> waiting for IO threads...");
>          dataAvailable.wait(3000);
>        }
>        dataAvailable.notifyAll();
>      }
>
> If the code is below. It seems better.
>
> // If we crossed the chunk threshold, wait for more space to be available
>      synchronized (dataAvailable) {
>        while (totalBuffered > maxHeapUsage) {
>          LOG.debug("Used " + totalBuffered + " bytes of buffered edits,
> waiting for IO threads...");
>          dataAvailable.wait(3000);
>        }
>        dataAvailable.notifyAll();
>      }
>
>
>
>
Mime
View raw message