hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vimal Jain <vkj...@gmail.com>
Subject Re: Region splits even when max Hstorefile is less then default value
Date Thu, 06 Mar 2014 13:22:50 GMT
Thanks Jean.


On Thu, Mar 6, 2014 at 6:41 PM, Jean-Marc Spaggiari <jean-marc@spaggiari.org
> wrote:

> Default split policy for 0.94.17 is
> IncreasingToUpperBoundRegionSplitPolicy. Since you did not set a specific
> policy in your config file, that's the one used.
>
> Comments from the class:
>  * Split size is the number of regions that are on this server that all are
>  * of the same table, cubed, times 2x the region flush size OR the maximum
>  * region split size, whichever is smaller.  For example, if the flush size
>  * is 128M, then after two flushes (256MB) we will split which will make
> two regions
>  * that will split when their size is 2^3 * 128M*2 = 2048M.  If one of
> these
>  * regions splits, then there are three regions and now the split size is
>  * 3^3 * 128M*2 =  6912M, and so on until we reach the configured
>  * maximum filesize and then from there on out, we'll use that.
>
> JM
>
>
> 2014-03-06 8:08 GMT-05:00 Vimal Jain <vkjk89@gmail.com>:
>
> > Hi,
> > I am using 0.94.17 and my config file is
> >
> >                 <property>
> >                 <name>hbase.rootdir</name>
> >                 <value>hdfs://10.14.24.19:9000/hbase</value>
> >                 </property>
> >                 <property>
> >                 <name>hbase.cluster.distributed</name>
> >                 <value>true</value>
> >                 </property>
> >                 <property>
> >                 <name>hbase.zookeeper.quorum</name>
> >                 <value>10.14.24.19</value>
> >                 </property>
> >                 <property>
> >                 <name>dfs.replication</name>
> >                 <value>2</value>
> >                 </property>
> >                 <property>
> >                 <name>hbase.zookeeper.property.clientPort</name>
> >                 <value>2181</value>
> >                 </property>
> >                 <property>
> >                 <name>hbase.zookeeper.property.dataDir</name>
> >                 <value>/home/hadoop/HbaseData/zookeeper</value>
> >                 </property>
> >                 <property>
> >                 <name>hbase.tmp.dir</name>
> >                 <value>/home/hadoop/Hbasetemp</value>
> >                 </property>
> >
> >
> >
> > On Thu, Mar 6, 2014 at 6:35 PM, Jean-Marc Spaggiari <
> > jean-marc@spaggiari.org
> > > wrote:
> >
> > > Hi Vimal,
> > >
> > > Which version of HBase do you use? Can you also share your config file?
> > >
> > > JM
> > >
> > >
> > > 2014-03-06 8:02 GMT-05:00 Vimal Jain <vkjk89@gmail.com>:
> > >
> > > > Hi Jean,
> > > > I am not sure about this.
> > > > Whats the default policy ?
> > > >
> > > >
> > > > On Thu, Mar 6, 2014 at 6:30 PM, Jean-Marc Spaggiari <
> > > > jean-marc@spaggiari.org
> > > > > wrote:
> > > >
> > > > > Hi Vimal,
> > > > >
> > > > > This is used only with ConstantSizeRegionSplitPolicy. Which split
> > > policy
> > > > do
> > > > > you have configured in your setup?
> > > > >
> > > > > JM
> > > > >
> > > > >
> > > > > 2014-03-06 7:55 GMT-05:00 Vimal Jain <vkjk89@gmail.com>:
> > > > >
> > > > > > Hi,
> > > > > > I am running 2 node hbase cluster atop HDFS.
> > > > > > I was simulating a heavy write process on Hbase for performance
> > > > analysis.
> > > > > > I found that my HstoreFile has not  reached
> > > > "hbase.hregion.max.filesize"
> > > > > > property ( which is by default 10G ) and yet my region is split
> > into
> > > 2
> > > > > > regions.
> > > > > >
> > > > > > *du -sh HadoopData/*
> > > > > > *3.2G HadoopData/*
> > > > > >
> > > > > > where HadoopData is my hadoop.tmp.dir ( place where hadoop stores
> > > data
> > > > ).
> > > > > >
> > > > > > Please help in understanding this.
> > > > > > --
> > > > > > Thanks and Regards,
> > > > > > Vimal Jain
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Thanks and Regards,
> > > > Vimal Jain
> > > >
> > >
> >
> >
> >
> > --
> > Thanks and Regards,
> > Vimal Jain
> >
>



-- 
Thanks and Regards,
Vimal Jain

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message