Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EB0D69D0E for ; Wed, 13 Jun 2012 05:35:47 +0000 (UTC) Received: (qmail 13809 invoked by uid 500); 13 Jun 2012 05:35:47 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 13600 invoked by uid 500); 13 Jun 2012 05:35:47 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 13500 invoked by uid 99); 13 Jun 2012 05:35:45 -0000 Received: from issues-vm.apache.org (HELO issues-vm) (140.211.11.160) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 13 Jun 2012 05:35:45 +0000 Received: from isssues-vm.apache.org (localhost [127.0.0.1]) by issues-vm (Postfix) with ESMTP id F1C441402B5 for ; Wed, 13 Jun 2012 05:35:44 +0000 (UTC) Date: Wed, 13 Jun 2012 05:35:44 +0000 (UTC) From: "nneverwei (JIRA)" To: issues@hbase.apache.org Message-ID: <1744308487.11175.1339565744992.JavaMail.jiratomcat@issues-vm> In-Reply-To: <953162019.46602.1339048825613.JavaMail.jiratomcat@issues-vm> Subject: [jira] [Commented] (HBASE-6185) Update javadoc for ConstantSizeRegionSplitPolicy class MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-6185?page=3Dcom.atlassian= .jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1329= 4165#comment-13294165 ]=20 nneverwei commented on HBASE-6185: ---------------------------------- {code:java} =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionS= plitPolicy.java=09(revision 1348721) +++ hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionS= plitPolicy.java=09(revision ) @@ -31,7 +31,9 @@ =20 /** * A split policy determines when a region should be split. - * {@see ConstantSizeRegionSplitPolicy} + * @see IncreasingToUpperBoundRegionSplitPolicy Default split policy since + * 0.94.0 + * @see ConstantSizeRegionSplitPolicy Default split policy before 0.94.0 */ @InterfaceAudience.Private public abstract class RegionSplitPolicy extends Configured { {code} =20 > Update javadoc for ConstantSizeRegionSplitPolicy class > ------------------------------------------------------ > > Key: HBASE-6185 > URL: https://issues.apache.org/jira/browse/HBASE-6185 > Project: HBase > Issue Type: Bug > Components: documentation > Affects Versions: 0.94.0 > Reporter: nneverwei > Fix For: 0.94.1 > > Attachments: HBASE-6185.patch, HBASE-6185.v2.patch > > > When using hbase0.94.0 we met a strange problem. > We config the 'hbase.hregion.max.filesize' to 100Gb (The recommed value t= o act as auto-split turn off).=20 > {code:xml} > > hbase.hregion.max.filesize > 107374182400 > > {code} > Then we keep putting datas into a table. > But when the data size far more less than 100Gb(about 500~600 uncompresse= d datas), the table auto splte to 2 regions... > I change the log4j config to DEBUG, and saw logs below: > {code} > 2012-06-07 10:30:52,161 INFO org.apache.hadoop.hbase.regionserver.HRegion= : Finished memstore flush of ~128.0m/134221272, currentsize=3D1.5m/1617744 = for region FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8.= in 3201ms, sequenceid=3D176387980, compaction requested=3Dfalse > 2012-06-07 10:30:52,161 DEBUG org.apache.hadoop.hbase.regionserver.Increa= singToUpperBoundRegionSplitPolicy: ShouldSplit because info size=3D13865741= 6, sizeToCheck=3D134217728, regionsWithCommonTable=3D1 > 2012-06-07 10:30:52,161 DEBUG org.apache.hadoop.hbase.regionserver.Incr= easingToUpperBoundRegionSplitPolicy: ShouldSplit because info size=3D138657= 416, sizeToCheck=3D134217728, regionsWithCommonTable=3D1 > 2012-06-07 10:30:52,240 DEBUG org.apache.hadoop.hbase.regionserver.Compac= tSplitThread: Split requested for FileStructIndex,,1339032525500.7b229abcd0= 785408251a579e9bdf49c8.. compaction_queue=3D(0:0), split_queue=3D0 > 2012-06-07 10:30:52,265 INFO org.apache.hadoop.hbase.regionserver.SplitTr= ansaction: Starting split of region FileStructIndex,,1339032525500.7b229abc= d0785408251a579e9bdf49c8. > 2012-06-07 10:30:52,265 DEBUG org.apache.hadoop.hbase.regionserver.SplitT= ransaction: regionserver:60020-0x137c4929efe0001 Creating ephemeral node fo= r 7b229abcd0785408251a579e9bdf49c8 in SPLITTING state > 2012-06-07 10:30:52,368 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:= regionserver:60020-0x137c4929efe0001 Attempting to transition node 7b229ab= cd0785408251a579e9bdf49c8 from RS_ZK_REGION_SPLITTING to RS_ZK_REGION_SPLIT= TING > 2012-06-07 10:30:52,382 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:= regionserver:60020-0x137c4929efe0001 Successfully transitioned node 7b229a= bcd0785408251a579e9bdf49c8 from RS_ZK_REGION_SPLITTING to RS_ZK_REGION_SPLI= TTING > 2012-06-07 10:30:52,410 DEBUG org.apache.hadoop.hbase.regionserver.HRegio= n: Closing FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8.= : disabling compactions & flushes > 2012-06-07 10:30:52,410 DEBUG org.apache.hadoop.hbase.regionserver.HRegio= nServer: NotServingRegionException; FileStructIndex,,1339032525500.7b229abc= d0785408251a579e9bdf49c8. is closing > 2012-06-07 10:30:52,411 DEBUG org.apache.hadoop.hbase.regionserver.HRegio= nServer: NotServingRegionException; FileStructIndex,,1339032525500.7b229abc= d0785408251a579e9bdf49c8. is closing > {code} > {color:red}IncreasingToUpperBoundRegionSplitPolicy: ShouldSplit because i= nfo size=3D138657416, sizeToCheck=3D134217728{color} > I did not config splitPolicy for hbase, so it means *IncreasingToUpperBou= ndRegionSplitPolicy is the default splitPolicy of 0.94.0* > After add > {code:xml} > > hbase.regionserver.region.split.policy > org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPo= licy > > {code} > autosplit did not happen again and everything goes well. > But we can still see javadoc on ConstantSizeRegionSplitPolicy, it says 'T= his is the default split policy'. Or even in the http://hbase.apache.org/bo= ok/regions.arch.html 9.7.4.1. Custom Split Policies, 'default split policy:= ConstantSizeRegionSplitPolicy.'. > Those may mistaken us that if we set hbase.hregion.max.filesize to 100Gb,= than the auto-split can be almost shutdown. > You may change those docs, and What more, in many scenerys, we actually n= eed to control split manually=EF=BC=88As you know when spliting the table a= re offline, reads and writes will fail=EF=BC=89=20 > =20 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrato= rs: https://issues.apache.org/jira/secure/ContactAdministrators!default.jsp= a For more information on JIRA, see: http://www.atlassian.com/software/jira