hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Hmaster crashes caused by splitting log.
Date Thu, 23 Jun 2011 03:48:18 GMT
I think you can place this condition for dataAvailable.notifyAll():
if (thrown == null) {
  dataAvailable.notifyAll();
}

2011/6/22 Gaojinchao <gaojinchao@huawei.com>

> Thanks, You are right.  :)
>
> I will test it.
>
>
> -----邮件原件-----
> 发件人: Ted Yu [mailto:yuzhihong@gmail.com]
> 发送时间: 2011年6月23日 11:00
> 收件人: dev@hbase.apache.org
> 主题: Re: Hmaster crashes caused by splitting log.
>
> I guess you have seen this call directly below the following code snippet:
>      checkForErrors();
> I think checking thrown should be kept in the condition.
>
> I assume we can lift the above call before dataAvailable.notifyAll()
>
> My 2 cents.
>
> On Wed, Jun 22, 2011 at 7:00 PM, Gaojinchao <gaojinchao@huawei.com> wrote:
>
> >
> > Because Master usually<javascript:void(0)> uses little memory. So its
> > memory is 4G.
> >
> > DFS block is 256M and hbase.regionserver.maxlogs is 32. One region server
> > can save max 8G Hlog.
> >
> > In my performance cluster(0.90.3), The Hmaster memory from 100 M up to 4G
> > when one region server crashed.
> >
> > I dug it and found the flow control does not work when write thread is
> > normal.
> >
> >
> > // If we crossed the chunk threshold, wait for more space to be available
> >      synchronized (dataAvailable) {
> >        while (totalBuffered > maxHeapUsage && thrown == null) {
> >          LOG.debug("Used " + totalBuffered + " bytes of buffered edits,
> > waiting for IO threads...");
> >          dataAvailable.wait(3000);
> >        }
> >        dataAvailable.notifyAll();
> >      }
> >
> > If the code is below. It seems better.
> >
> > // If we crossed the chunk threshold, wait for more space to be available
> >      synchronized (dataAvailable) {
> >        while (totalBuffered > maxHeapUsage) {
> >          LOG.debug("Used " + totalBuffered + " bytes of buffered edits,
> > waiting for IO threads...");
> >          dataAvailable.wait(3000);
> >        }
> >        dataAvailable.notifyAll();
> >      }
> >
> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message