hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From NNever <nnever...@gmail.com>
Subject Re: Region autoSplit when not reach 'hbase.hregion.max.filesize' ?
Date Thu, 07 Jun 2012 02:25:06 GMT
The rowkey is just like an UUID. no order.  And there is an coprocessor
trigger datas to another 2 index-tables when doing put...

Thanks. yours
NN

2012/6/7 Michael Segel <michael_segel@hotmail.com>

> Just out of curiosity, describe the data?
> Sorted?
> The more we know, the easier it is to help... Also, can you recheck your
> math ?
>
>
> Sent from my iPhone
>
> On Jun 6, 2012, at 6:17 PM, "NNever" <nneverwei@gmail.com> wrote:
>
> > It comes again. I truncate the table, and put about 10million datas into
> it
> > last night.
> > The table auto-split to 4, each has about 3Gb storefileUncompressedSize.
> >
> > I grep the log and out but nothing about the split.
> >
> > the logs are as below:
> > 2012-06-06 19:31:15,402 WARN org.apache.hadoop.ipc.HBaseServer:
> > (responseTooSlow):
> > {"processingtimems":10296,"call":"next(1511657428305700194, 1), rpc
> > version=1, client version=29, methodsFingerPrint=-1508511443","client":"
> > 192.168.1.145:46456
> >
> ","starttimems":1338982265104,"queuetimems":0,"class":"HRegionServer","responsesize":6,"method":"next"}
> > 2012-06-06 19:31:15,606 WARN org.apache.hadoop.ipc.HBaseServer:
> > (responseTooSlow):
> > {"processingtimems":10842,"call":"next(-2954106234340837837, 1), rpc
> > version=1, client version=29, methodsFingerPrint=-1508511443","client":"
> > 192.168.1.145:46456
> >
> ","starttimems":1338982264763,"queuetimems":1,"class":"HRegionServer","responsesize":6,"method":"next"}
> > 2012-06-06 19:31:29,795 WARN org.apache.hadoop.ipc.HBaseServer:
> > (responseTooSlow):
> > {"processingtimems":10668,"call":"next(2455689470981850756, 1), rpc
> > version=1, client version=29, methodsFingerPrint=-1508511443","client":"
> > 192.168.1.145:46456
> >
> ","starttimems":1338982279126,"queuetimems":0,"class":"HRegionServer","responsesize":6,"method":"next"}
> > 2012-06-06 20:24:54,157 WARN org.apache.hadoop.ipc.HBaseServer:
> > (responseTooSlow):
> >
> {"processingtimems":2920400,"call":"multi(org.apache.hadoop.hbase.client.MultiAction@6b39de40
> ),
> > rpc version=1, client version=29,
> methodsFingerPrint=-1508511443","client":"
> > 192.168.1.145:46456
> >
> ","starttimems":1338982573756,"queuetimems":0,"class":"HRegionServer","responsesize":0,"method":"multi"}
> > 2012-06-06 20:24:54,251 WARN org.apache.hadoop.ipc.HBaseServer: IPC
> Server
> > Responder, call multi(org.apache.hadoop.hbase.client.MultiAction@6b39de40
> ),
> > rpc version=1, client version=29, methodsFingerPrint=-1508511443 from
> > 192.168.1.145:46456: output error
> > 2012-06-06 20:24:54,294 WARN org.apache.hadoop.ipc.HBaseServer: IPC
> Server
> > handler 2159 on 60020 caught a ClosedChannelException, this means that
> the
> > server was processing a request but the client went away. The error
> message
> > was: null
> > 2012-06-06 20:25:00,868 WARN org.apache.hadoop.ipc.HBaseServer:
> > (responseTooSlow):
> {"processingtimems":2927114,"call":"multi(org.apache.hado
> >
> > You can see on 19:31:29 the log stop for about 1 hour, It may doing split
> > there.(this is the regionserver.log)
> > And in the regionServer.out I cannot find any information about split,
> Only
> > lots of 'org.apache.hadoop.hbase.NotServingRegionException' when split.
> No
> > log about start doing split and why do split.
> >
> > logs are too large to upload somewhere.
> >
> > I'll dig into it....It really confuse me...
> >
> > Thanks, yours
> > NN
> >
> >
> > 2012/6/6 NNever <nneverwei@gmail.com>
> >
> >> I'll. I changed the log level.
> >> Putting datas and waiting for the strange split now   :).....
> >>
> >> Yours,
> >> NN
> >>
> >> 2012/6/6 dong.yajun <dongtalk@gmail.com>
> >>
> >> Hi NNever
> >>>
> >>> If you find any issues, please let us known, thanks.
> >>>
> >>>
> >>> On Wed, Jun 6, 2012 at 5:09 PM, NNever <nneverwei@gmail.com> wrote:
> >>>
> >>>> I'm sorry, the log4j now is WARN, not INFO
> >>>>
> >>>> 2012/6/6 NNever <nneverwei@gmail.com>
> >>>>
> >>>>> We currently run in INFO mode.
> >>>>> It actully did the split, but I cannot find any logs about this
> split.
> >>>>> I will change the log4j to DEBUG, if got any log valuable, I will
> >>> paste
> >>>>> here...
> >>>>>
> >>>>> Thanks Ram,
> >>>>> NN
> >>>>>
> >>>>> 2012/6/6 Ramkrishna.S.Vasudevan <ramkrishna.vasudevan@huawei.com>
> >>>>>
> >>>>> You have any logs corresponding to this?
> >>>>>>
> >>>>>> Regards
> >>>>>> Ram
> >>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: NNever [mailto:nneverwei@gmail.com]
> >>>>>>> Sent: Wednesday, June 06, 2012 2:12 PM
> >>>>>>> To: user@hbase.apache.org
> >>>>>>> Subject: Region autoSplit when not reach
> >>> 'hbase.hregion.max.filesize'
> >>>> ?
> >>>>>>>
> >>>>>>> The 'hbase.hregion.max.filesize' are set to 100G (The recommed
> >>> value
> >>>> to
> >>>>>>> act
> >>>>>>> as auto-split turn off). And there is a table, we keep put
datas
> >>> into
> >>>>>>> it.
> >>>>>>> When the storefileUncompressedSizeMB reached about 1Gb,
the region
> >>>> auto
> >>>>>>> splite to 2.
> >>>>>>> I don't know how it happened? 1G is far more less than
> >>> max.filesize-
> >>>>>>> 100G.
> >>>>>>> So if there is any possible scenery that will
> >>>>>>> ignore hbase.hregion.max.filesize and do split?
> >>>>>>>
> >>>>>>> How can I totally shutdown the autoSplit?
> >>>>>>>
> >>>>>>>
> >>>>>>> -----------------
> >>>>>>> Best regards,
> >>>>>>> NN
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> *Ric Dong *
> >>> Newegg Ecommerce, MIS department
> >>>
> >>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message