Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of yuzhihong@gmail.com designates
 209.85.213.53 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <8645A306C824894AA8EE0B9C91D62AE074799669@G9W0747.americas.hpqcorp.net>
References: 
 <8645A306C824894AA8EE0B9C91D62AE074799669@G9W0747.americas.hpqcorp.net>
Date: Thu, 20 Nov 2014 09:32:21 -0800
Message-ID: 
 <CALte62zkYfypWxXxHNUGpyU22nb3MPoD73tFGMiT+6wr4oi0Tw@mail.gmail.com>
Subject: Re: how to explain read/write performance change after modifying the
 hfile.block.cache.size?
From: Ted Yu <yuzhihong@gmail.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=001a113994feefde4105084db506

--001a113994feefde4105084db506
Content-Type: text/plain; charset=UTF-8

When block cache size increases from 0 to 0.4, the amount of heap given to
memstore decreases. This would slow down the writes.
Please see:
http://hbase.apache.org/book.html#store.memstore

For your second question, see this thread:
http://search-hadoop.com/m/DHED4TEvBy1/lars+hbase+hflush&subj=Re+Clarifications+on+HBase+Durability

Cheers

On Thu, Nov 20, 2014 at 8:05 AM, Liu, Ming (HPIT-GADSC) <ming.liu2@hp.com>
wrote:

> Hello, all,
>
> I am playing with YCSB to test HBase performance. I am using HBase 0.98.5.
> I tried to adjust the hfile.block.cache.size to see the difference, when I
> set hfile.block.cache.size to 0, read performance is very bad, but write
> performance is very very very good....; when I set hfile.block.cache.size
> to 0.4, read is better, but write performance drop dramatically. I disable
> the client side writebuffer already.
> This is hard to understand for me:
> The HBase guide just said hfile.block.cache.size setting is about how much
> memory used as block cache used by StoreFile. I have no idea of how HBase
> works internally. Typically, it is easy to understand that increase the
> size of cache should help the read, but why it will harm the write
> operation? The write performance down from 30,000 to 4,000 for your
> reference, just by changing the hfile.block.cache.size from 0 to 0.4.
> Could anyone give me a brief explanation about this observation or give me
> some advices about what to study to understand what is block cache used for?
>
> Another question: HBase write will first write to WAL then to memstore.
> Will the write to WAL go to disk directly before hbase write memstore, a
> sync operation or it is possible that write to WAL is still buffered
> somewhere when hbase put the data into the memstore?
>
> Reading src code may cost me months, so a kindly reply will help me a
> lot... ...
> Thanks very much!
>
> Best Regards,
> Ming
>

--001a113994feefde4105084db506--