hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Leffel (JIRA)" <j...@apache.org>
Subject [jira] Created: (HBASE-617) Region Server not Exiting after failing
Date Tue, 06 May 2008 20:12:55 GMT
Region Server not Exiting after failing
---------------------------------------

                 Key: HBASE-617
                 URL: https://issues.apache.org/jira/browse/HBASE-617
             Project: Hadoop HBase
          Issue Type: Bug
    Affects Versions: 0.1.2
            Reporter: Daniel Leffel
            Priority: Minor


If a region server was overloaded, the process never exists. Also, this causes an hbase shutdown
to wait indefinitely (as called from the shell script). 


This is the thread dump from the regionserver:
--------------------------------------------
Full thread dump Java HotSpot(TM) 64-Bit Server VM (1.5.0_14-b03 mixed mode):

"IPC Server handler 9 on 60020" daemon prio=1 tid=0x00002aab1c29eb40 nid=0x45db waiting on
condition [0x0000000042835000..0x0000000042835c80]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:118)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:716)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:746)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1076)
        at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:184)
        at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:256)
        at org.apache.hadoop.hbase.HLog.rollWriter(HLog.java:219)
        at org.apache.hadoop.hbase.HLog.append(HLog.java:390)
        - locked <0x00002aaab70634d0> (a java.lang.Integer)
        at org.apache.hadoop.hbase.HRegion.update(HRegion.java:1602)
        at org.apache.hadoop.hbase.HRegion.batchUpdate(HRegion.java:1405)
        at org.apache.hadoop.hbase.HRegionServer.batchUpdate(HRegionServer.java:1544)
        at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:585)
        at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:901)

"regionserver/0:0:0:0:0:0:0:0:60020.cacheFlusher" daemon prio=1 tid=0x00002aab1c1cb800 nid=0x45c8
waiting for monitor entry [0x0000000041522000..0x0000000041522b00]
        at org.apache.hadoop.hbase.HLog.completeCacheFlush(HLog.java:481)
        - waiting to lock <0x00002aaab70634d0> (a java.lang.Integer)
        at org.apache.hadoop.hbase.HRegion.internalFlushcache(HRegion.java:1103)
        at org.apache.hadoop.hbase.HRegion.flushcache(HRegion.java:998)
        at org.apache.hadoop.hbase.HRegionServer$Flusher.flushRegion(HRegionServer.java:447)
        - locked <0x00002aaab70681f8> (a java.util.HashSet)
        at org.apache.hadoop.hbase.HRegionServer$Flusher.run(HRegionServer.java:390)

"regionserver/0:0:0:0:0:0:0:0:60020.logRoller" daemon prio=1 tid=0x00002aab1c1cc2b0 nid=0x45c7
waiting on condition [0x0000000041421000..0x0000000041421a80]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:118)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:716)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:746)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1076)
        at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:184)
        at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:256)
        at org.apache.hadoop.hbase.HLog.rollWriter(HLog.java:219)
        at org.apache.hadoop.hbase.HRegionServer$LogRoller.run(HRegionServer.java:618)
        - locked <0x00002aaab70632b8> (a java.lang.Integer)

"org.apache.hadoop.dfs.DFSClient$LeaseChecker@353c375" daemon prio=1 tid=0x00002aab1c0893c0
nid=0x45c5 waiting on condition [0x000000004121f000..0x000000004121fd80]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.hadoop.dfs.DFSClient$LeaseChecker.run(DFSClient.java:605)
        at java.lang.Thread.run(Thread.java:595)

"org.apache.hadoop.io.ObjectWritable Connection Culler" daemon prio=1 tid=0x00002aab1c10ad00
nid=0x45c3 waiting on condition [0x000000004101d000..0x000000004101dc80]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.hadoop.ipc.Client$ConnectionCuller.run(Client.java:423)

"org.apache.hadoop.hbase.io.HbaseObjectWritable Connection Culler" daemon prio=1 tid=0x000000005093f200
nid=0x45b5 waiting on condition [0x0000000040e1b000..0x0000000040e1bc80]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.hadoop.ipc.Client$ConnectionCuller.run(Client.java:423)

"DestroyJavaVM" prio=1 tid=0x00002aab1c00d930 nid=0x458f waiting on condition [0x0000000000000000..0x00007fff1680e7f0]

"regionserver/0:0:0:0:0:0:0:0:60020" prio=1 tid=0x00002aab1c00dfe0 nid=0x45b4 waiting for
monitor entry [0x0000000040d1a000..0x0000000040d1ac00]
        at org.apache.hadoop.hbase.HRegionServer.run(HRegionServer.java:867)
        - waiting to lock <0x00002aaab70632b8> (a java.lang.Integer)
        at java.lang.Thread.run(Thread.java:595)

"Low Memory Detector" daemon prio=1 tid=0x00000000505e9c90 nid=0x45b2 runnable [0x0000000000000000..0x0000000000000000]

"CompilerThread1" daemon prio=1 tid=0x00000000505e8060 nid=0x45b1 waiting on condition [0x0000000000000000..0x0000000040a16360]

"CompilerThread0" daemon prio=1 tid=0x00000000505e6ab0 nid=0x45b0 waiting on condition [0x0000000000000000..0x00000000409156e0]

"AdapterThread" daemon prio=1 tid=0x00000000505e5350 nid=0x45af waiting on condition [0x0000000000000000..0x0000000000000000]

"Signal Dispatcher" daemon prio=1 tid=0x00000000505e4070 nid=0x45ae runnable [0x0000000000000000..0x0000000000000000]

"Finalizer" daemon prio=1 tid=0x00000000505cfef0 nid=0x45ad in Object.wait() [0x0000000040613000..0x0000000040613c80]
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00002aaab70bc3d8> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:120)
        - locked <0x00002aaab70bc3d8> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:136)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=1 tid=0x00000000505cf6c0 nid=0x45ac in Object.wait() [0x0000000040512000..0x0000000040512c00]
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00002aaab70b3ac8> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:474)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
        - locked <0x00002aaab70b3ac8> (a java.lang.ref.Reference$Lock)

"VM Thread" prio=1 tid=0x00000000505cb1d0 nid=0x45ab runnable 

"GC task thread#0 (ParallelGC)" prio=1 tid=0x000000005051e4d0 nid=0x45a9 runnable 

"GC task thread#1 (ParallelGC)" prio=1 tid=0x000000005051f350 nid=0x45aa runnable 

"VM Periodic Task Thread" prio=1 tid=0x0000000050515430 nid=0x45b3 waiting on condition 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message