hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <michael_se...@hotmail.com>
Subject Re: Region autoSplit when not reach 'hbase.hregion.max.filesize' ?
Date Thu, 07 Jun 2012 02:19:18 GMT
Just out of curiosity, describe the data?
Sorted?
The more we know, the easier it is to help... Also, can you recheck your math ?


Sent from my iPhone

On Jun 6, 2012, at 6:17 PM, "NNever" <nneverwei@gmail.com> wrote:

> It comes again. I truncate the table, and put about 10million datas into it
> last night.
> The table auto-split to 4, each has about 3Gb storefileUncompressedSize.
> 
> I grep the log and out but nothing about the split.
> 
> the logs are as below:
> 2012-06-06 19:31:15,402 WARN org.apache.hadoop.ipc.HBaseServer:
> (responseTooSlow):
> {"processingtimems":10296,"call":"next(1511657428305700194, 1), rpc
> version=1, client version=29, methodsFingerPrint=-1508511443","client":"
> 192.168.1.145:46456
> ","starttimems":1338982265104,"queuetimems":0,"class":"HRegionServer","responsesize":6,"method":"next"}
> 2012-06-06 19:31:15,606 WARN org.apache.hadoop.ipc.HBaseServer:
> (responseTooSlow):
> {"processingtimems":10842,"call":"next(-2954106234340837837, 1), rpc
> version=1, client version=29, methodsFingerPrint=-1508511443","client":"
> 192.168.1.145:46456
> ","starttimems":1338982264763,"queuetimems":1,"class":"HRegionServer","responsesize":6,"method":"next"}
> 2012-06-06 19:31:29,795 WARN org.apache.hadoop.ipc.HBaseServer:
> (responseTooSlow):
> {"processingtimems":10668,"call":"next(2455689470981850756, 1), rpc
> version=1, client version=29, methodsFingerPrint=-1508511443","client":"
> 192.168.1.145:46456
> ","starttimems":1338982279126,"queuetimems":0,"class":"HRegionServer","responsesize":6,"method":"next"}
> 2012-06-06 20:24:54,157 WARN org.apache.hadoop.ipc.HBaseServer:
> (responseTooSlow):
> {"processingtimems":2920400,"call":"multi(org.apache.hadoop.hbase.client.MultiAction@6b39de40),
> rpc version=1, client version=29, methodsFingerPrint=-1508511443","client":"
> 192.168.1.145:46456
> ","starttimems":1338982573756,"queuetimems":0,"class":"HRegionServer","responsesize":0,"method":"multi"}
> 2012-06-06 20:24:54,251 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server
> Responder, call multi(org.apache.hadoop.hbase.client.MultiAction@6b39de40),
> rpc version=1, client version=29, methodsFingerPrint=-1508511443 from
> 192.168.1.145:46456: output error
> 2012-06-06 20:24:54,294 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server
> handler 2159 on 60020 caught a ClosedChannelException, this means that the
> server was processing a request but the client went away. The error message
> was: null
> 2012-06-06 20:25:00,868 WARN org.apache.hadoop.ipc.HBaseServer:
> (responseTooSlow): {"processingtimems":2927114,"call":"multi(org.apache.hado
> 
> You can see on 19:31:29 the log stop for about 1 hour, It may doing split
> there.(this is the regionserver.log)
> And in the regionServer.out I cannot find any information about split, Only
> lots of 'org.apache.hadoop.hbase.NotServingRegionException' when split. No
> log about start doing split and why do split.
> 
> logs are too large to upload somewhere.
> 
> I'll dig into it....It really confuse me...
> 
> Thanks, yours
> NN
> 
> 
> 2012/6/6 NNever <nneverwei@gmail.com>
> 
>> I'll. I changed the log level.
>> Putting datas and waiting for the strange split now   :).....
>> 
>> Yours,
>> NN
>> 
>> 2012/6/6 dong.yajun <dongtalk@gmail.com>
>> 
>> Hi NNever
>>> 
>>> If you find any issues, please let us known, thanks.
>>> 
>>> 
>>> On Wed, Jun 6, 2012 at 5:09 PM, NNever <nneverwei@gmail.com> wrote:
>>> 
>>>> I'm sorry, the log4j now is WARN, not INFO
>>>> 
>>>> 2012/6/6 NNever <nneverwei@gmail.com>
>>>> 
>>>>> We currently run in INFO mode.
>>>>> It actully did the split, but I cannot find any logs about this split.
>>>>> I will change the log4j to DEBUG, if got any log valuable, I will
>>> paste
>>>>> here...
>>>>> 
>>>>> Thanks Ram,
>>>>> NN
>>>>> 
>>>>> 2012/6/6 Ramkrishna.S.Vasudevan <ramkrishna.vasudevan@huawei.com>
>>>>> 
>>>>> You have any logs corresponding to this?
>>>>>> 
>>>>>> Regards
>>>>>> Ram
>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: NNever [mailto:nneverwei@gmail.com]
>>>>>>> Sent: Wednesday, June 06, 2012 2:12 PM
>>>>>>> To: user@hbase.apache.org
>>>>>>> Subject: Region autoSplit when not reach
>>> 'hbase.hregion.max.filesize'
>>>> ?
>>>>>>> 
>>>>>>> The 'hbase.hregion.max.filesize' are set to 100G (The recommed
>>> value
>>>> to
>>>>>>> act
>>>>>>> as auto-split turn off). And there is a table, we keep put datas
>>> into
>>>>>>> it.
>>>>>>> When the storefileUncompressedSizeMB reached about 1Gb, the region
>>>> auto
>>>>>>> splite to 2.
>>>>>>> I don't know how it happened? 1G is far more less than
>>> max.filesize-
>>>>>>> 100G.
>>>>>>> So if there is any possible scenery that will
>>>>>>> ignore hbase.hregion.max.filesize and do split?
>>>>>>> 
>>>>>>> How can I totally shutdown the autoSplit?
>>>>>>> 
>>>>>>> 
>>>>>>> -----------------
>>>>>>> Best regards,
>>>>>>> NN
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> *Ric Dong *
>>> Newegg Ecommerce, MIS department
>>> 
>> 
>> 

Mime
View raw message