hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <la...@apache.org>
Subject Re: Slow Get Performance (or how many disk I/O does it take for one non-cached read?)
Date Sun, 02 Feb 2014 06:33:59 GMT
Hmm... Interesting. I expected there to be a better improvement from smaller blocks.

So it's really just IOPS (and block size does not matter), in which case, yes, HBase checksum
will save you 50% IOPS for each data block (and since index blocks are cache) it'll save 50%
total IOPS.



________________________________
 From: Jan Schellenberger <leipzig3@gmail.com>
To: user@hbase.apache.org 
Sent: Saturday, February 1, 2014 9:38 PM
Subject: Re: Slow Get Performance (or how many disk I/O does it take for one non-cached read?)
 

I've experimented with the block size.  Here are results:
4k - 60reads/sec  - 1.2GB totalStaticIndexSize
8k - 80reads/sec  - 660MB totalStaticIndexSize
16k - 90reads/sec  - 330MB totalStaticIndexSize
and previously
64k - 80reads/sec - ~100mb totalStaticIndexSize


Also, I turned off caching and you are correct, the index blocks seem to be
cached always - the blockCachedSize grows until it reaches
totalStaticIndexSize and then stops growing.  If you turn caching on, it
will grow until the maxHeap * blockCacheSize (.4 in my case).

I saw no performance difference between caching off/on so I guess off is
fine.

Yes, I always do a major_compact before testing.

I think this probably concludes my question - I will try to upgrade to a
newer hbase version to get the CRC32/HDFS check fix and we will probably
have to evaluate SSD's.  

Cheers, everyone.




--
View this message in context: http://apache-hbase.679495.n3.nabble.com/Slow-Get-Performance-or-how-many-disk-I-O-does-it-take-for-one-non-cached-read-tp4055545p4055582.html

Sent from the HBase User mailing list archive at Nabble.com.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message