hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matteo Bertozzi <theo.berto...@gmail.com>
Subject Re: Deadlock in WALProcedureStore
Date Tue, 23 Feb 2016 21:58:35 GMT
yeah same as HBASE-14947, the 2nd lock on the store tracker is gone with
that change

Matteo


On Tue, Feb 23, 2016 at 1:50 PM, Nick Dimiduk <ndimiduk@gmail.com> wrote:

> Heya,
>
> I ran into a deadlock today (create table via phoenix) and I'm wondering if
> its the same as HBASE-14947 or if this is a new issue. This is in 1.1.2.
>
> Thanks,
> Nick
>
> Found one Java-level deadlock:
> =============================
> "WALProcedureStoreSyncThread":
>   waiting to lock monitor 0x00007fd4e1b4f548 (object 0x00000006d5328a38, a
> org.apache.hadoop.hbase.procedure2.store.ProcedureStoreTracker),
>   which is held by "B.defaultRpcServer.handler=14,queue=2,port=56060"
> "B.defaultRpcServer.handler=14,queue=2,port=56060":
>   waiting for ownable synchronizer 0x00000006d52e4728, (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync),
>   which is held by "WALProcedureStoreSyncThread"
>
> Java stack information for the threads listed above:
> ===================================================
> "WALProcedureStoreSyncThread":
>         at
>
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:726)
>         - waiting to lock <0x00000006d5328a38> (a
> org.apache.hadoop.hbase.procedure2.store.ProcedureStoreTracker)
>         at
>
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:676)
>         at
>
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.checkAndTryRoll(WALProcedureStore.java:655)
>         at
>
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.syncLoop(WALProcedureStore.java:525)
>         at
>
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.access$000(WALProcedureStore.java:66)
>         at
>
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore$1.run(WALProcedureStore.java:176)
> "B.defaultRpcServer.handler=14,queue=2,port=56060":
>
>         at sun.misc.Unsafe.park(Native Method)
>
>         - parking to wait for  <0x00000006d52e4728> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
>         at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>
>         at
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
>
>         at
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
>
>         at
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
>
>         at
>
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
>
>         at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
>
>         at
>
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:722)
>
>         at
>
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:676)
>
>         at
>
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.checkAndTryRoll(WALProcedureStore.java:655)
>
>         at
>
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.insert(WALProcedureStore.java:355)
>
>         - locked <0x00000006d5328a38> (a
> org.apache.hadoop.hbase.procedure2.store.ProcedureStoreTracker)
>
>         at
>
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:524)
>
>         at
> org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1459)
>
>         at
>
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:422)
>
>         at
>
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:48502)
>
>         at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114)
>
>         at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101)
>
>         at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
>
>         at
> org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
>
>         at java.lang.Thread.run(Thread.java:745)
>
>
> Found 1 deadlock.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message