hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Question on changing block settings...
Date Tue, 03 Aug 2010 22:54:50 GMT
Inline.

J-D

On Tue, Aug 3, 2010 at 2:44 PM, Vidhyashankar Venkataraman
<vidhyash@yahoo-inc.com> wrote:
> A few issues I have been observing on changing block settings:
>
>  1.  What happens if we change the block size of a column family on an already populated
database? Will this not throw apps on db out of whack because of compression and Hfile index
which depend on block size?  So, once the db is populated, the user should be prevented from
altering the block size..

You can, when the files are compacted they will be re-written with the
new block size. When you read files that weren't compacted, we just
read the index and figure the blocks so there's nothing wrong there
either. It works like that too when you open a file that isn't
compressed on a table that was just changed to be LZOed or else.

>
>
>   2.   I had disabled the block cache through Java API and through hbase shell, yet
I get stats like the following: I had gone through this with Stack and he pointed out that
this could be a bug:
>
> 2010-07-27 14:13:40,852 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats:
Sizes: Total=1950.0773MB (2044804168), Free=449.7541MB (471601336), Max=2399.8313MB (2516405504),
Counts: Blocks=1914, Access=1304510, Hit=1914, Miss=1302596, Evictions=3300, Evicted=788515,
Ratios: Hit Ratio=0.1467217574827373%, Miss Ratio=99.8532772064209%, Evicted/Run=238.943939208984382010-07-27
14:14:40,860 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes: Total=1950.0773MB
(2044804168), Free=449.7541MB (471601336), Max=2399.8313MB (2516405504), Counts: Blocks=1914,
Access=1306103, Hit=1914, Miss=1304189, Evictions=3300, Evicted=788515, Ratios: Hit Ratio=0.14654280385002494%,
Miss Ratio=99.85345602035522%, Evicted/Run=238.943939208984382010-07-27 14:15:40,853 DEBUG
org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes: Total=1950.0773MB (2044804168),
Free=449.7541MB (471601336), Max=2399.8313MB (2516405504), Counts: Blocks=1914, Access=1307783,
Hit=1914, Miss=1305869, Evictions=3300, Evicted=788515, Ratios: Hit Ratio=0.14635456027463078%,
Miss Ratio=99.8536467552185%, Evicted/Run=238.94393920898438

If you didn't reboot the region server processes, then the stats won't
be cleared (since other tables could be using the block cache). Is it
the case?

Mime
View raw message