hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: disabling table timed out
Date Tue, 23 Nov 2010 13:34:02 GMT
I tried to restart hbase. But the region server identified by ZNode
/hbase/root-region-server declared that it is not serving root region:

2010-11-23 13:26:49,617 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
locateRegionInMeta attempt 1 of 3 failed; retrying after sleep of 5000
because: Timed out trying to locate root region because:
org.apache.hadoop.hbase.NotServingRegionException: -ROOT-,,0
        at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2274)
        at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionInfo(HRegionServer.java:1711)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at
org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
        at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:998)

2010-11-23 13:26:54,622 DEBUG
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.202.50.111:60020
2010-11-23 13:26:54,624 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Root region
location changed. Sleeping.
2010-11-23 13:26:59,626 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Wake. Retry
finding root region.
2010-11-23 13:26:59,629 DEBUG
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.202.50.111:60020
2010-11-23 13:26:59,630 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Root region
location changed. Sleeping.
2010-11-23 13:27:04,632 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Wake. Retry
finding root region.
2010-11-23 13:27:04,635 DEBUG
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.202.50.111:60020

What should I do next ?

Thanks

On Tue, Nov 23, 2010 at 1:37 AM, Lars George <lars.george@gmail.com> wrote:

> Hi Ted,
>
> So one of the regions is not being released? Could you try and see
> from .META. which is still deployed and use the shell's "close_region"
> to close it while looking at the master and region server logs to see
> what is going on? Maybe best if you switch the RS to DEBUG level
> logging first to get some info?
>
> Lars
>
> On Tue, Nov 23, 2010 at 8:25 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> > Hi
> > We use 0.20.6
> >
> > I tried to disable packageindex table. From master log:
> >
> > 2010-11-23 07:21:06,326 DEBUG
> > org.apache.hadoop.hbase.master.ChangeTableState: Adding region
> > packageindex,CC7E6FEA4CDCF19C6F4AC9BB51EF6A33,1290230596786 to setClosing
> > list for us01-ciqps1-grid10.carrieriq.com,60020,1290493641949
> > 2010-11-23 07:21:06,326 DEBUG
> > org.apache.hadoop.hbase.master.ChangeTableState: Adding region
> > packageindex,F2A18967F48C9FDA9C23BF9A8210ED17,1290230394345 to setClosing
> > list for us01-ciqps1-grid11.carrieriq.com,60020,1290493641228
> > 2010-11-23 07:21:06,326 DEBUG
> > org.apache.hadoop.hbase.master.ChangeTableState: Adding region
> > packageindex,E8FA713B2F030EF012E5AB0A641CB1DB,1290230356969 to setClosing
> > list for us01-ciqps1-grid11.carrieriq.com,60020,1290493641228
> > 2010-11-23 07:21:06,327 DEBUG
> > org.apache.hadoop.hbase.master.ChangeTableState: Adding region
> > packageindex,5B10CA26DCAEFBFF4A63DB7D0432D628,1290229869191 to setClosing
> > list for us01-ciqps1-grid12.carrieriq.com,60020,1290493641232
> > 2010-11-23 07:21:20,178 INFO
> org.apache.hadoop.hbase.master.ServerManager:
> > 15 region servers, 0 dead, average load 123.66666666666667
> > 2010-11-23 07:21:20,252 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.rootScanner scanning meta region {server:
> 10.202.50.111:60020,
> > regionname: -ROOT-,,0, startKey: <>}
> > 2010-11-23 07:21:20,257 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.rootScanner scan of 1 row(s) of meta region {server:
> > 10.202.50.111:60020, regionname: -ROOT-,,0, startKey: <>} complete
> > 2010-11-23 07:21:22,838 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.metaScanner scanning meta region {server:
> 10.202.50.101:60020,
> > regionname: .META.,,1, startKey: <>}
> > 2010-11-23 07:21:24,731 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.metaScanner scan of 2086 row(s) of meta region {server:
> > 10.202.50.101:60020, regionname: .META.,,1, startKey: <>} complete
> > 2010-11-23 07:21:24,731 INFO org.apache.hadoop.hbase.master.BaseScanner:
> All
> > 1 .META. region(s) scanned
> >
> > But I always got:
> > hbase(main):004:0> disable 'packageindex'
> > NativeException: org.apache.hadoop.hbase.RegionException: Retries
> exhausted,
> > it took too long to wait for the table packageindex to be disabled.
> >
> > What should I do to disable the table ?
> >
> > Thanks
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message