hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From NNever <nnever...@gmail.com>
Subject Re: Region autoSplit when not reach 'hbase.hregion.max.filesize' ?
Date Thu, 07 Jun 2012 01:16:18 GMT
It comes again. I truncate the table, and put about 10million datas into it
last night.
The table auto-split to 4, each has about 3Gb storefileUncompressedSize.

I grep the log and out but nothing about the split.

the logs are as below:
2012-06-06 19:31:15,402 WARN org.apache.hadoop.ipc.HBaseServer:
(responseTooSlow):
{"processingtimems":10296,"call":"next(1511657428305700194, 1), rpc
version=1, client version=29, methodsFingerPrint=-1508511443","client":"
192.168.1.145:46456
","starttimems":1338982265104,"queuetimems":0,"class":"HRegionServer","responsesize":6,"method":"next"}
2012-06-06 19:31:15,606 WARN org.apache.hadoop.ipc.HBaseServer:
(responseTooSlow):
{"processingtimems":10842,"call":"next(-2954106234340837837, 1), rpc
version=1, client version=29, methodsFingerPrint=-1508511443","client":"
192.168.1.145:46456
","starttimems":1338982264763,"queuetimems":1,"class":"HRegionServer","responsesize":6,"method":"next"}
2012-06-06 19:31:29,795 WARN org.apache.hadoop.ipc.HBaseServer:
(responseTooSlow):
{"processingtimems":10668,"call":"next(2455689470981850756, 1), rpc
version=1, client version=29, methodsFingerPrint=-1508511443","client":"
192.168.1.145:46456
","starttimems":1338982279126,"queuetimems":0,"class":"HRegionServer","responsesize":6,"method":"next"}
2012-06-06 20:24:54,157 WARN org.apache.hadoop.ipc.HBaseServer:
(responseTooSlow):
{"processingtimems":2920400,"call":"multi(org.apache.hadoop.hbase.client.MultiAction@6b39de40),
rpc version=1, client version=29, methodsFingerPrint=-1508511443","client":"
192.168.1.145:46456
","starttimems":1338982573756,"queuetimems":0,"class":"HRegionServer","responsesize":0,"method":"multi"}
2012-06-06 20:24:54,251 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server
Responder, call multi(org.apache.hadoop.hbase.client.MultiAction@6b39de40),
rpc version=1, client version=29, methodsFingerPrint=-1508511443 from
192.168.1.145:46456: output error
2012-06-06 20:24:54,294 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server
handler 2159 on 60020 caught a ClosedChannelException, this means that the
server was processing a request but the client went away. The error message
was: null
2012-06-06 20:25:00,868 WARN org.apache.hadoop.ipc.HBaseServer:
(responseTooSlow): {"processingtimems":2927114,"call":"multi(org.apache.hado

You can see on 19:31:29 the log stop for about 1 hour, It may doing split
there.(this is the regionserver.log)
And in the regionServer.out I cannot find any information about split, Only
lots of 'org.apache.hadoop.hbase.NotServingRegionException' when split. No
log about start doing split and why do split.

logs are too large to upload somewhere.

I'll dig into it....It really confuse me...

Thanks, yours
NN


2012/6/6 NNever <nneverwei@gmail.com>

> I'll. I changed the log level.
> Putting datas and waiting for the strange split now   :).....
>
> Yours,
> NN
>
> 2012/6/6 dong.yajun <dongtalk@gmail.com>
>
> Hi NNever
>>
>> If you find any issues, please let us known, thanks.
>>
>>
>> On Wed, Jun 6, 2012 at 5:09 PM, NNever <nneverwei@gmail.com> wrote:
>>
>> > I'm sorry, the log4j now is WARN, not INFO
>> >
>> > 2012/6/6 NNever <nneverwei@gmail.com>
>> >
>> > > We currently run in INFO mode.
>> > > It actully did the split, but I cannot find any logs about this split.
>> > > I will change the log4j to DEBUG, if got any log valuable, I will
>> paste
>> > > here...
>> > >
>> > > Thanks Ram,
>> > > NN
>> > >
>> > > 2012/6/6 Ramkrishna.S.Vasudevan <ramkrishna.vasudevan@huawei.com>
>> > >
>> > > You have any logs corresponding to this?
>> > >>
>> > >> Regards
>> > >> Ram
>> > >>
>> > >> > -----Original Message-----
>> > >> > From: NNever [mailto:nneverwei@gmail.com]
>> > >> > Sent: Wednesday, June 06, 2012 2:12 PM
>> > >> > To: user@hbase.apache.org
>> > >> > Subject: Region autoSplit when not reach
>> 'hbase.hregion.max.filesize'
>> > ?
>> > >> >
>> > >> > The 'hbase.hregion.max.filesize' are set to 100G (The recommed
>> value
>> > to
>> > >> > act
>> > >> > as auto-split turn off). And there is a table, we keep put datas
>> into
>> > >> > it.
>> > >> > When the storefileUncompressedSizeMB reached about 1Gb, the region
>> > auto
>> > >> > splite to 2.
>> > >> > I don't know how it happened? 1G is far more less than
>> max.filesize-
>> > >> > 100G.
>> > >> > So if there is any possible scenery that will
>> > >> > ignore hbase.hregion.max.filesize and do split?
>> > >> >
>> > >> > How can I totally shutdown the autoSplit?
>> > >> >
>> > >> >
>> > >> > -----------------
>> > >> > Best regards,
>> > >> > NN
>> > >>
>> > >>
>> > >
>> >
>>
>>
>>
>> --
>> *Ric Dong *
>> Newegg Ecommerce, MIS department
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message