hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Heng Chen (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-15900) RS stuck in get lock of HStore
Date Fri, 27 May 2016 07:38:13 GMT
Heng Chen created HBASE-15900:
---------------------------------

             Summary: RS stuck in get lock of HStore
                 Key: HBASE-15900
                 URL: https://issues.apache.org/jira/browse/HBASE-15900
             Project: HBase
          Issue Type: Bug
            Reporter: Heng Chen
         Attachments: dump.txt

It happens on my production cluster when i run MR job.  I save the dump.txt from this RS webUI.

Many threads stuck here:
{code}
Thread 133 (B.defaultRpcServer.handler=94,queue=4,port=16020):
   32   State: WAITING
   31   Blocked count: 477816
   30   Waited count: 535255
   29   Waiting on java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@6447ba67
   28   Stack:
   27     sun.misc.Unsafe.park(Native Method)
   26     java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   25     java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
   24     java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
   23     java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
   22     java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
   21     org.apache.hadoop.hbase.regionserver.HStore.add(HStore.java:666)
   20     org.apache.hadoop.hbase.regionserver.HRegion.applyFamilyMapToMemstore(HRegion.java:3621)
   19     org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3038)
   18     org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2793)
   17     org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2735)
   16     org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:692)
   15     org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:654)
   14     org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2029)
   13     org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32213)
   12     org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2112)
   11     org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101)
   10     org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
    9     org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
    8     java.lang.Thread.run(Thread.java:745)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message