hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergey Belousov <sergey.belou...@gmail.com>
Subject Re: Splitting table from HBase shell using script
Date Sat, 18 Apr 2015 01:40:46 GMT
Hm
That what I am not sure I understand ...
It is not like I requesting location of regions every time (or asking to
move it). Here is more details what I am doing ...

1. I get regions for the table and store it in array.
2. I get array of split points to create by splits_to_create -
current_splits_for_the_table
3. I run *splits_to_create*.each |split| do split 'table', split sleep(2)
end

It also appears that all regions created on one (random) region server if I
run it on master or if I run it on region server it will be created on that
region...
When I have 5 sec pause between - I do not have this error...
BTW I also tried with balance_switch false (interesting naming :)) - no
difference. Same would be to run balancer before so it will sits on time
out to the next run while I splitting unless I run it after I am done (what
I do)

So why would shell be confused when I run bunch of 'split' commands for the
table ?

I would try to stay within running splits in one shell and in one session
if possible...

Thank you

On Fri, Apr 17, 2015 at 5:57 PM, Esteban Gutierrez <esteban@cloudera.com>
wrote:

> Hi Sergey,
>
> I think this is what is happening: the hbase shell should be caching the
> region locations, if the balancer is on and you run those split commands it
> is very likely you will be getting that error since the regions have moved
> to other RSs. The easiest solution I think is to run the splits on
> different hbase shell processes so every time it will use a new connection.
> You could also try to close the connections or purge the cached locations
> from the hbase shell process.
>
> thanks,
> esteban.
>
> --
> Cloudera, Inc.
>
>
> On Fri, Apr 17, 2015 at 12:43 PM, Sergey Belousov <
> sergey.belousov@gmail.com
> > wrote:
>
> > Thank you Esteban for reply
> >
> > see inline
> >
> > On Apr 17, 2015 12:50 PM, "Esteban Gutierrez" <esteban@cloudera.com>
> > wrote:
> > >
> > > Sergey,
> > >
> > > My first question would be if you are turning off the HBase balancer
> > before
> > > splitting this region and how long between locating the region and then
> > > splitting the region has passed.
> >
> >  I do not disable balancer before.
> >
> > Also, are you splitting multiple regions
> > > per run of your ruby script or just one?
> >
> > I do create multiple splits for the table at the time. But it all
> > sequential through shell
> >
> > like
> >
> > split 'table', 'splitkey1'
> > sleep(2)
> > split 'table', 'splitkey1'
> > sleep(2)
> > split 'table', 'splitkey1'
> > etc..
> >
> >
> > >
> > > thanks!
> > > esteban.
> > >
> > > --
> > > Cloudera, Inc.
> > >
> > >
> > > On Fri, Apr 17, 2015 at 9:41 AM, Sergey Belousov <
> > sergey.belousov@gmail.com>
> > > wrote:
> > >
> > > > Hi all
> > > >
> > > > I was looking into doing some splitting for the table (sort of in
> > > > production so I can not use SPLITS on create time) using 'split'
> > command
> > > > from HBase shell. (0.98.9-hadoop2)
> > > > I have simple JRuby script where I just run 'split'
> > > > 'table_name','split_key' command using hbase shell
> /tmp/split_table.rb
> > > >
> > > > Time from time I have this error coming up
> > > >
> > > > ERROR: org.apache.hadoop.hbase.NotServingRegionException: Region
> > > >
> > > > [****************eaten by
> > > > mouse***************],1429257507107.82bfbd974d36db11075e4ef1da7abfed.
> > > > is not online on ******************,60020,1429256987509
> > > >         at
> > > >
> >
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2780)
> > > >         at
> > > >
> >
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:4337)
> > > >         at
> > > >
> >
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:4042)
> > > >         at
> > > >
> >
> >
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:20170)
> > > >         at
> > org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2029)
> > > >         at
> > org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
> > > >         at
> > > >
> >
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:112)
> > > >         at
> > > > org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:92)
> > > >         at java.lang.Thread.run(Thread.java:745)
> > > >
> > > >
> > > >
> > > > Even thou sleep(2) help mostly it's not guaranty (especially on live
> > > > cluster) and not the solution I would like.
> > > >
> > > > So my question is what would be proper way of checking that I am good
> > to
> > > > run next split command?
> > > >
> > > >
> > > > Thank you
> > > > SB
> > > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message