hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sebastian Bauer (JIRA)" <j...@apache.org>
Subject [jira] Created: (HBASE-3190) Problem with disabling and droping table
Date Wed, 03 Nov 2010 08:22:24 GMT
Problem with disabling and droping table
----------------------------------------

                 Key: HBASE-3190
                 URL: https://issues.apache.org/jira/browse/HBASE-3190
             Project: HBase
          Issue Type: Bug
            Reporter: Sebastian Bauer
             Fix For: 0.90.0


Table disabling was interrupted by kill -9 all part of hbase and now we cannot do anything
with this table, disabling doesn't show any exception:
hbase(main):019:0> disable 'NGolden_CTU'
0 row(s) in 0.0250 seconds


but droping show this:
hbase(main):020:0> drop 'NGolden_CTU'   

ERROR: org.apache.hadoop.hbase.TableNotDisabledException: org.apache.hadoop.hbase.TableNotDisabledException:
NGolden_CTU
        at org.apache.hadoop.hbase.master.HMaster.checkTableModifiable(HMaster.java:861)
        at org.apache.hadoop.hbase.master.handler.TableEventHandler.<init>(TableEventHandler.java:52)
        at org.apache.hadoop.hbase.master.handler.DeleteTableHandler.<init>(DeleteTableHandler.java:42)
        at org.apache.hadoop.hbase.master.HMaster.deleteTable(HMaster.java:779)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1025)

Here is some help for this command:
          Drop the named table. Table must first be disabled. If table has
          more than one region, run a major compaction on .META.:

          hbase> major_compact ".META."

after this nothing strange is in logs

when we restart hbase we get this:

2010-11-03 08:56:37,892 DEBUG org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler:
Processing open of NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.
                                                       
2010-11-03 08:56:37,892 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: regionserver:60020-0x12c10b5fb780005
Attempting to transition node 0c8579e52b0ea3f2dab5b6a857ad030b from M_ZK_REGION_OFFLINE to
RS_ZK_REGION_OPENING                                                         
2010-11-03 08:56:37,892 ERROR org.apache.hadoop.hbase.executor.EventHandler: Caught throwable
while processing event M_RS_OPEN_REGION                                                  
                                                                                         
     
java.lang.NullPointerException                                                           
                                                                                         
                                                                                         
         
        at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:75)         
                                                                                         
                                                                                         
         
        at org.apache.hadoop.hbase.executor.RegionTransitionData.fromBytes(RegionTransitionData.java:198)
                                                                                         
                                                                                   
        at org.apache.hadoop.hbase.zookeeper.ZKAssign.transitionNode(ZKAssign.java:669)  
                                                                                         
                                                                                         
         
        at org.apache.hadoop.hbase.zookeeper.ZKAssign.transitionNodeOpening(ZKAssign.java:549)
                                                                                         
                                                                                         
    
        at org.apache.hadoop.hbase.zookeeper.ZKAssign.transitionNodeOpening(ZKAssign.java:542)
                                                                                         
                                                                                         
    
        at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.transitionZookeeperOfflineToOpening(OpenRegionHandler.java:208)
                                                                                         
                                                   
        at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:89)
                                                                                         
                                                                                
        at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:151)      
                                                                                         
                                                                                         
         
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
                                                                                         
                                                                                         
    
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
                                                                                         
                                                                                         
        
        at java.lang.Thread.run(Thread.java:619)                                         
                                    



Other logs with this region:

hbase@db2a:logs$ grep "0c8579e52b0ea3f2dab5b6a857ad030b" hbase-hbase-master-db2a.goldenline.pl.log

2010-11-03 08:54:02,575 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:60000-0x12c10b5fb780002
Async create of unassigned node for 0c8579e52b0ea3f2dab5b6a857ad030b with OFFLINE state
2010-11-03 08:54:03,555 DEBUG org.apache.hadoop.hbase.master.AssignmentManager$CreateUnassignedAsyncCallback:
rs=NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.
state=OFFLINE, ts=1288770842575, server=db2a.goldenline.pl,60020,1288770551154
2010-11-03 08:54:03,777 DEBUG org.apache.hadoop.hbase.master.AssignmentManager$ExistsUnassignedAsyncCallback:
rs=NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.
state=OFFLINE, ts=1288770842575
2010-11-03 08:54:42,836 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in
transition timed out:  NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.
state=PENDING_OPEN, ts=1288770843777
2010-11-03 08:54:42,836 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has
been PENDING_OPEN for too long, reassigning region=NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.
2010-11-03 08:54:42,836 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE;
was=NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.
state=PENDING_OPEN, ts=1288770843777
2010-11-03 08:54:42,836 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: No previous
transition plan was found (or we are ignoring an existing plan) for NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.
so generated a random one; hri=NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.,
src=, dest=db2b.goldenline.pl,60020,1288770553679; 2 (online=2, exclude=null) available servers
2010-11-03 08:54:42,836 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Assigning
region NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.
to db2b.goldenline.pl,60020,1288770553679
2010-11-03 08:56:12,824 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_OPENING,
server=db2a.goldenline.pl,60020,1288770551154, region=0c8579e52b0ea3f2dab5b6a857ad030b
2010-11-03 08:56:12,975 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_OPENED,
server=db2a.goldenline.pl,60020,1288770551154, region=0c8579e52b0ea3f2dab5b6a857ad030b
2010-11-03 08:56:12,976 DEBUG org.apache.hadoop.hbase.master.handler.OpenedRegionHandler:
Handling OPENED event for 0c8579e52b0ea3f2dab5b6a857ad030b; deleting unassigned node
2010-11-03 08:56:12,976 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:60000-0x12c10b5fb780002
Deleting existing unassigned node for 0c8579e52b0ea3f2dab5b6a857ad030b that is in expected
state RS_ZK_REGION_OPENED
2010-11-03 08:56:12,978 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:60000-0x12c10b5fb780002
Successfully deleted unassigned node for region 0c8579e52b0ea3f2dab5b6a857ad030b in expected
state RS_ZK_REGION_OPENED
2010-11-03 08:56:13,011 DEBUG org.apache.hadoop.hbase.master.handler.OpenedRegionHandler:
Opened region NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.
on db2a.goldenline.pl,60020,1288770551154





hbase@db2a:logs$ grep "0c8579e52b0ea3f2dab5b6a857ad030b" hbase-hbase-regionserver-db2a.goldenline.pl.log

2010-11-03 08:54:04,470 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Received
request to open region: NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.
2010-11-03 08:56:12,770 DEBUG org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler:
Processing open of NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.
2010-11-03 08:56:12,770 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: regionserver:60020-0x12c10b5fb780001
Attempting to transition node 0c8579e52b0ea3f2dab5b6a857ad030b from M_ZK_REGION_OFFLINE to
RS_ZK_REGION_OPENING
2010-11-03 08:56:12,819 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: regionserver:60020-0x12c10b5fb780001
Successfully transitioned node 0c8579e52b0ea3f2dab5b6a857ad030b from M_ZK_REGION_OFFLINE to
RS_ZK_REGION_OPENING
2010-11-03 08:56:12,819 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region:
REGION => {NAME => 'NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.',
STARTKEY => '3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E', ENDKEY => '3065-d_2010_9_11_47D955785DEC3CE95377D70F67BA5ABC',
ENCODED => 0c8579e52b0ea3f2dab5b6a857ad030b, TABLE => {{NAME => 'NGolden_CTU', FAMILIES
=> [{NAME => 'c', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION
=> 'LZO', VERSIONS => '1', TTL => '-1', BLOCKSIZE => '65536', IN_MEMORY =>
'false', BLOCKCACHE => 'true'}]}}
2010-11-03 08:56:12,819 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Instantiated NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.
2010-11-03 08:56:12,939 INFO org.apache.hadoop.hbase.regionserver.StoreFile$Reader: Loaded
row bloom filter metadata for hdfs://db2a:50001/hbase/NGolden_CTU/0c8579e52b0ea3f2dab5b6a857ad030b/c/561636411807463667
2010-11-03 08:56:12,939 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded hdfs://db2a:50001/hbase/NGolden_CTU/0c8579e52b0ea3f2dab5b6a857ad030b/c/561636411807463667,
isReference=false, isBulkLoadResult=false, seqid=1820373362, majorCompaction=true
2010-11-03 08:56:12,939 INFO org.apache.hadoop.hbase.regionserver.HRegion: Onlined NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.;
next sequenceid=1820373363
2010-11-03 08:56:12,939 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: regionserver:60020-0x12c10b5fb780001
Attempting to transition node 0c8579e52b0ea3f2dab5b6a857ad030b from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENING
2010-11-03 08:56:12,970 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: regionserver:60020-0x12c10b5fb780001
Successfully transitioned node 0c8579e52b0ea3f2dab5b6a857ad030b from RS_ZK_REGION_OPENING
to RS_ZK_REGION_OPENING
2010-11-03 08:56:12,971 INFO org.apache.hadoop.hbase.catalog.MetaEditor: Updated row NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.
in region .META.,,1 with server=db2a.goldenline.pl:60020, startcode=1288770551154
2010-11-03 08:56:12,971 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: regionserver:60020-0x12c10b5fb780001
Attempting to transition node 0c8579e52b0ea3f2dab5b6a857ad030b from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENED
2010-11-03 08:56:12,973 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: regionserver:60020-0x12c10b5fb780001
Successfully transitioned node 0c8579e52b0ea3f2dab5b6a857ad030b from RS_ZK_REGION_OPENING
to RS_ZK_REGION_OPENED
2010-11-03 08:56:12,973 DEBUG org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler:
Opened NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.




hbase@db2b:logs$  grep "0c8579e52b0ea3f2dab5b6a857ad030b" hbase-hbase-regionserver-db2b.goldenline.pl.log
2010-11-03 08:54:42,840 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Received
request to open region: NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.
2010-11-03 08:56:37,892 DEBUG org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler:
Processing open of NGolden_CTU,3065-d_2010_10_14_245FF1A15F4E236002ED3AB651BAB97E,1288046281444.0c8579e52b0ea3f2dab5b6a857ad030b.
2010-11-03 08:56:37,892 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: regionserver:60020-0x12c10b5fb780005
Attempting to transition node 0c8579e52b0ea3f2dab5b6a857ad030b from M_ZK_REGION_OFFLINE to
RS_ZK_REGION_OPENING


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message