hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From NNever <nnever...@gmail.com>
Subject Re: Region autoSplit when not reach 'hbase.hregion.max.filesize' ?
Date Thu, 07 Jun 2012 02:55:34 GMT
On 0.94.0, In class RegionSplitPolicy, I saw you
use IncreasingToUpperBoundRegionSplitPolicy as DEFAULT_SPLIT_POLICY_CLASS.
But the javadoc tells that defalut policy is ConstantSizeRegionSplitPolicy.

So is the  DEFAULT_SPLIT_POLICY_CLASS wrong or the javadoc has not update
yet?

Yours,
NN

2012/6/7 NNever <nneverwei@gmail.com>

> So  IncreasingToUpperBoundRegionSplitPolicy  will  do split when size
> reach (square region-num)* flushSize until reach the maxfileSize.
> We didn't config splitPolicy, will hbase0.94 use  IncreasingToUpper
> BoundRegionSplitPolicy  as default?
>
>
> 2012/6/7 NNever <nneverwei@gmail.com>
>
>> Finally I change the log4j conf and try again, the split log comes
>> out.......
>>
>> 2012-06-07 10:30:52,161 INFO
>> org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of
>> ~128.0m/134221272, currentsize=1.5m/1617744 for region
>> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8. in 3201ms,
>> sequenceid=176387980, compaction requested=false
>> 2012-06-07 10:30:52,161 DEBUG org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy:
>> ShouldSplit because info size=138657416, sizeToCheck=134217728,
>> regionsWithCommonTable=1
>> 2012-06-07 10:30:52,161 DEBUG org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy:
>> ShouldSplit because info size=138657416, sizeToCheck=134217728,
>> regionsWithCommonTable=1
>> 2012-06-07 10:30:52,240 DEBUG
>> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Split requested
>> for FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8..
>>  compaction_queue=(0:0), split_queue=0
>> 2012-06-07 10:30:52,265 INFO
>> org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of
>> region FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8.
>> 2012-06-07 10:30:52,265 DEBUG
>> org.apache.hadoop.hbase.regionserver.SplitTransaction:
>> regionserver:60020-0x137c4929efe0001 Creating ephemeral node for
>> 7b229abcd0785408251a579e9bdf49c8 in SPLITTING state
>> 2012-06-07 10:30:52,368 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
>> regionserver:60020-0x137c4929efe0001 Attempting to transition node
>> 7b229abcd0785408251a579e9bdf49c8 from RS_ZK_REGION_SPLITTING to
>> RS_ZK_REGION_SPLITTING
>> 2012-06-07 10:30:52,382 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
>> regionserver:60020-0x137c4929efe0001 Successfully transitioned node
>> 7b229abcd0785408251a579e9bdf49c8 from RS_ZK_REGION_SPLITTING to
>> RS_ZK_REGION_SPLITTING
>> 2012-06-07 10:30:52,410 DEBUG
>> org.apache.hadoop.hbase.regionserver.HRegion: Closing
>> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8.: disabling
>> compactions & flushes
>> 2012-06-07 10:30:52,410 DEBUG
>> org.apache.hadoop.hbase.regionserver.HRegionServer:
>> NotServingRegionException;
>> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8. is closing
>> 2012-06-07 10:30:52,411 DEBUG
>> org.apache.hadoop.hbase.regionserver.HRegionServer:
>> NotServingRegionException;
>> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8. is closing
>>
>>
>> Best regards,
>> NN
>>
>>
>> 2012/6/7 NNever <nneverwei@gmail.com>
>>
>>> We use hbase 0.94.0, running on a single mechine.
>>> The hbase-site.xml:
>>>
>>> <configuration>
>>>> <property>
>>>> <name>hbase.rootdir</name>
>>>> <value>hdfs://xxxxx/hbase</value>
>>>>  </property>
>>>> <property>
>>>> <name>hbase.cluster.distributed</name>
>>>>  <value>true</value>
>>>> </property>
>>>> <property>
>>>>  <name>hbase.zookeeper.quorum</name>
>>>> <value>xxxxx</value>
>>>> </property>
>>>>  <property>
>>>> <name>hbase.zookeeper.property.dataDir</name>
>>>> <value>/mybk/zookeeper</value>
>>>>  </property>
>>>> <property>
>>>> <name>hbase.hregion.max.filesize</name>
>>>>  <value>107374182400</value>
>>>> </property>
>>>> <property>
>>>>  <name>zookeeper.session.timeout</name>
>>>> <value>60000</value>
>>>> </property>
>>>>  <property>
>>>> <name>hbase.regionserver.handler.count</name>
>>>> <value>4000</value>
>>>>  </property>
>>>> <property>
>>>> <name>hbase.client.write.buffer</name>
>>>>  <value>1048576</value>
>>>> </property>
>>>> <property>
>>>>  <name>hbase.client.scanner.caching</name>
>>>> <value>10</value>
>>>> </property>
>>>> </configuration>
>>>
>>>
>>>
>>>
>>> 2012/6/7 NNever <nneverwei@gmail.com>
>>>
>>>> The rowkey is just like an UUID. no order.  And there is an coprocessor
>>>> trigger datas to another 2 index-tables when doing put...
>>>>
>>>> Thanks. yours
>>>> NN
>>>>
>>>>
>>>> 2012/6/7 Michael Segel <michael_segel@hotmail.com>
>>>>
>>>>> Just out of curiosity, describe the data?
>>>>> Sorted?
>>>>> The more we know, the easier it is to help... Also, can you recheck
>>>>> your math ?
>>>>>
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>> On Jun 6, 2012, at 6:17 PM, "NNever" <nneverwei@gmail.com> wrote:
>>>>>
>>>>> > It comes again. I truncate the table, and put about 10million datas
>>>>> into it
>>>>> > last night.
>>>>> > The table auto-split to 4, each has about 3Gb
>>>>> storefileUncompressedSize.
>>>>> >
>>>>> > I grep the log and out but nothing about the split.
>>>>> >
>>>>> > the logs are as below:
>>>>> > 2012-06-06 19:31:15,402 WARN org.apache.hadoop.ipc.HBaseServer:
>>>>> > (responseTooSlow):
>>>>> > {"processingtimems":10296,"call":"next(1511657428305700194, 1),
rpc
>>>>> > version=1, client version=29,
>>>>> methodsFingerPrint=-1508511443","client":"
>>>>> > 192.168.1.145:46456
>>>>> >
>>>>> ","starttimems":1338982265104,"queuetimems":0,"class":"HRegionServer","responsesize":6,"method":"next"}
>>>>> > 2012-06-06 19:31:15,606 WARN org.apache.hadoop.ipc.HBaseServer:
>>>>> > (responseTooSlow):
>>>>> > {"processingtimems":10842,"call":"next(-2954106234340837837, 1),
rpc
>>>>> > version=1, client version=29,
>>>>> methodsFingerPrint=-1508511443","client":"
>>>>> > 192.168.1.145:46456
>>>>> >
>>>>> ","starttimems":1338982264763,"queuetimems":1,"class":"HRegionServer","responsesize":6,"method":"next"}
>>>>> > 2012-06-06 19:31:29,795 WARN org.apache.hadoop.ipc.HBaseServer:
>>>>> > (responseTooSlow):
>>>>> > {"processingtimems":10668,"call":"next(2455689470981850756, 1),
rpc
>>>>> > version=1, client version=29,
>>>>> methodsFingerPrint=-1508511443","client":"
>>>>> > 192.168.1.145:46456
>>>>> >
>>>>> ","starttimems":1338982279126,"queuetimems":0,"class":"HRegionServer","responsesize":6,"method":"next"}
>>>>> > 2012-06-06 20:24:54,157 WARN org.apache.hadoop.ipc.HBaseServer:
>>>>> > (responseTooSlow):
>>>>> >
>>>>> {"processingtimems":2920400,"call":"multi(org.apache.hadoop.hbase.client.MultiAction@6b39de40
>>>>> ),
>>>>> > rpc version=1, client version=29,
>>>>> methodsFingerPrint=-1508511443","client":"
>>>>> > 192.168.1.145:46456
>>>>> >
>>>>> ","starttimems":1338982573756,"queuetimems":0,"class":"HRegionServer","responsesize":0,"method":"multi"}
>>>>> > 2012-06-06 20:24:54,251 WARN org.apache.hadoop.ipc.HBaseServer:
IPC
>>>>> Server
>>>>> > Responder, call
>>>>> multi(org.apache.hadoop.hbase.client.MultiAction@6b39de40),
>>>>> > rpc version=1, client version=29, methodsFingerPrint=-1508511443
from
>>>>> > 192.168.1.145:46456: output error
>>>>> > 2012-06-06 20:24:54,294 WARN org.apache.hadoop.ipc.HBaseServer:
IPC
>>>>> Server
>>>>> > handler 2159 on 60020 caught a ClosedChannelException, this means
>>>>> that the
>>>>> > server was processing a request but the client went away. The error
>>>>> message
>>>>> > was: null
>>>>> > 2012-06-06 20:25:00,868 WARN org.apache.hadoop.ipc.HBaseServer:
>>>>> > (responseTooSlow):
>>>>> {"processingtimems":2927114,"call":"multi(org.apache.hado
>>>>> >
>>>>> > You can see on 19:31:29 the log stop for about 1 hour, It may doing
>>>>> split
>>>>> > there.(this is the regionserver.log)
>>>>> > And in the regionServer.out I cannot find any information about
>>>>> split, Only
>>>>> > lots of 'org.apache.hadoop.hbase.NotServingRegionException' when
>>>>> split. No
>>>>> > log about start doing split and why do split.
>>>>> >
>>>>> > logs are too large to upload somewhere.
>>>>> >
>>>>> > I'll dig into it....It really confuse me...
>>>>> >
>>>>> > Thanks, yours
>>>>> > NN
>>>>> >
>>>>> >
>>>>> > 2012/6/6 NNever <nneverwei@gmail.com>
>>>>> >
>>>>> >> I'll. I changed the log level.
>>>>> >> Putting datas and waiting for the strange split now   :).....
>>>>> >>
>>>>> >> Yours,
>>>>> >> NN
>>>>> >>
>>>>> >> 2012/6/6 dong.yajun <dongtalk@gmail.com>
>>>>> >>
>>>>> >> Hi NNever
>>>>> >>>
>>>>> >>> If you find any issues, please let us known, thanks.
>>>>> >>>
>>>>> >>>
>>>>> >>> On Wed, Jun 6, 2012 at 5:09 PM, NNever <nneverwei@gmail.com>
>>>>> wrote:
>>>>> >>>
>>>>> >>>> I'm sorry, the log4j now is WARN, not INFO
>>>>> >>>>
>>>>> >>>> 2012/6/6 NNever <nneverwei@gmail.com>
>>>>> >>>>
>>>>> >>>>> We currently run in INFO mode.
>>>>> >>>>> It actully did the split, but I cannot find any
logs about this
>>>>> split.
>>>>> >>>>> I will change the log4j to DEBUG, if got any log
valuable, I will
>>>>> >>> paste
>>>>> >>>>> here...
>>>>> >>>>>
>>>>> >>>>> Thanks Ram,
>>>>> >>>>> NN
>>>>> >>>>>
>>>>> >>>>> 2012/6/6 Ramkrishna.S.Vasudevan <ramkrishna.vasudevan@huawei.com
>>>>> >
>>>>> >>>>>
>>>>> >>>>> You have any logs corresponding to this?
>>>>> >>>>>>
>>>>> >>>>>> Regards
>>>>> >>>>>> Ram
>>>>> >>>>>>
>>>>> >>>>>>> -----Original Message-----
>>>>> >>>>>>> From: NNever [mailto:nneverwei@gmail.com]
>>>>> >>>>>>> Sent: Wednesday, June 06, 2012 2:12 PM
>>>>> >>>>>>> To: user@hbase.apache.org
>>>>> >>>>>>> Subject: Region autoSplit when not reach
>>>>> >>> 'hbase.hregion.max.filesize'
>>>>> >>>> ?
>>>>> >>>>>>>
>>>>> >>>>>>> The 'hbase.hregion.max.filesize' are set
to 100G (The recommed
>>>>> >>> value
>>>>> >>>> to
>>>>> >>>>>>> act
>>>>> >>>>>>> as auto-split turn off). And there is a
table, we keep put
>>>>> datas
>>>>> >>> into
>>>>> >>>>>>> it.
>>>>> >>>>>>> When the storefileUncompressedSizeMB reached
about 1Gb, the
>>>>> region
>>>>> >>>> auto
>>>>> >>>>>>> splite to 2.
>>>>> >>>>>>> I don't know how it happened? 1G is far
more less than
>>>>> >>> max.filesize-
>>>>> >>>>>>> 100G.
>>>>> >>>>>>> So if there is any possible scenery that
will
>>>>> >>>>>>> ignore hbase.hregion.max.filesize and do
split?
>>>>> >>>>>>>
>>>>> >>>>>>> How can I totally shutdown the autoSplit?
>>>>> >>>>>>>
>>>>> >>>>>>>
>>>>> >>>>>>> -----------------
>>>>> >>>>>>> Best regards,
>>>>> >>>>>>> NN
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>
>>>>> >>>>
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>> --
>>>>> >>> *Ric Dong *
>>>>> >>> Newegg Ecommerce, MIS department
>>>>> >>>
>>>>> >>
>>>>> >>
>>>>>
>>>>
>>>>
>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message