hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liu, Ming (HPIT-GADSC)" <ming.l...@hp.com>
Subject how to explain read/write performance change after modifying the hfile.block.cache.size?
Date Thu, 20 Nov 2014 16:05:52 GMT
Hello, all,

I am playing with YCSB to test HBase performance. I am using HBase 0.98.5. I tried to adjust
the hfile.block.cache.size to see the difference, when I set hfile.block.cache.size to 0,
read performance is very bad, but write performance is very very very good....; when I set
hfile.block.cache.size to 0.4, read is better, but write performance drop dramatically. I
disable the client side writebuffer already.
This is hard to understand for me:
The HBase guide just said hfile.block.cache.size setting is about how much memory used as
block cache used by StoreFile. I have no idea of how HBase works internally. Typically, it
is easy to understand that increase the size of cache should help the read, but why it will
harm the write operation? The write performance down from 30,000 to 4,000 for your reference,
just by changing the hfile.block.cache.size from 0 to 0.4.
Could anyone give me a brief explanation about this observation or give me some advices about
what to study to understand what is block cache used for?

Another question: HBase write will first write to WAL then to memstore. Will the write to
WAL go to disk directly before hbase write memstore, a sync operation or it is possible that
write to WAL is still buffered somewhere when hbase put the data into the memstore?

Reading src code may cost me months, so a kindly reply will help me a lot... ...
Thanks very much!

Best Regards,
Ming

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message