hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Marc Spaggiari <jean-m...@spaggiari.org>
Subject Re: HBase 0.98.1 split policy change?
Date Wed, 09 Apr 2014 19:39:57 GMT
Ok. From the logs, I can clearly see that my settings are not getting use:

2014-04-09 15:35:03,737 DEBUG [MemStoreFlusher.0]
regionserver.IncreasingToUpperBoundRegionSplitPolicy: ShouldSplit because
info size=2242479890, sizeToCheck=2147483648, regionsWithCommonTable=2

I will try to figure why and report.

JM



2014-04-09 15:25 GMT-04:00 Jean-Marc Spaggiari <jean-marc@spaggiari.org>:

> Hi,
>
> I try to chance my HBase 0.98.1 split policy to be
> ConstantSizeRegionSplitPolicy so I have updated my hbase-site.xml do be
> this:
>
>   <property>
>     <name>hbase.regionserver.region.split.policy</name>
>
> <value>org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy</value>
>   </property>
>
> Then on my table I have altered to get this where regions size should be
> 100GB:
> hbase(main):002:0> describe 'TestTable'
> DESCRIPTION
>
>  'TestTable', {TABLE_ATTRIBUTES => {MAX_FILESIZE => '1073741824000'},
> {NAME => 'info', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW',
> REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'
> 2147483647', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'false',
> BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
> 'true'}
>
> 1 row(s) in 0.2480 seconds
>
> However, I'm still getting multiple small regions with I try randomWrite
> into the table:
> hadoop@hbasetest1:~$ bin/hadoop fs -du -h /hbase/data/default/TestTable
> 14/04/09 15:08:13 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 320    /hbase/data/default/TestTable/.tabledesc
> 0      /hbase/data/default/TestTable/.tmp
> 1.6 G  /hbase/data/default/TestTable/1ad732fefba8ab6080820fb235fd5180
> 2.0 G  /hbase/data/default/TestTable/585b3e11d64cabcff332a2e293a37fee
> 1.5 G  /hbase/data/default/TestTable/6e8c24a2cc03051bb8a0990e22b8ec21
> 1.5 G  /hbase/data/default/TestTable/73fc51f6d6d75e57be29a1f7d54ef1df
> 1.4 G  /hbase/data/default/TestTable/ac69f15956bf413805982439a05527fd
> 2.5 G  /hbase/data/default/TestTable/cfb60af42b92fd9c2abfab80306d257c
>
> With the split policy setting and the alter, I will have expected to have
> a single region of few GB and not multiple regions. I checked inthe WebUI
> for the config and I can see my splitpolicy entry. What did I missed? I
> checked the ConstantSizeRegionSplitPolicy in 0.98 and it uses MAX_FILESIZE
> from the table, and hbase.hregion.max.filesize which is not setup for me.
> Any idea?
>
> JM
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message