pulsar-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] rdhabalia opened a new pull request #2745: Fix: deadlock on blocking ml operation
Date Mon, 08 Oct 2018 07:29:17 GMT
rdhabalia opened a new pull request #2745: Fix: deadlock on blocking ml operation
URL: https://github.com/apache/pulsar/pull/2745
 
 
   ### Motivation
   
   Found a deadlock in broker with below jstack.
   [Jstack](https://gist.github.com/rdhabalia/9cffa5005716d077a8a767a0cc6afd1b)
   
   expireMessages thread is waiting on topic-map and ml-operation couldn't complete which
created a deadlock at broker.
   
   ```
   Thread 17133: (state = BLOCKED)
    - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
    - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=175 (Compiled
frame)
    - java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt() @bci=1,
line=836 (Compiled frame)
    - java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(int)
@bci=72, line=997 (Compiled frame)
    - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(int)
@bci=24, line=1304 (Compiled frame)
    - java.util.concurrent.CountDownLatch.await() @bci=5, line=231 (Compiled frame)
    - org.apache.bookkeeper.mledger.impl.ManagedCursorImpl.getNthEntry(int, org.apache.bookkeeper.mledger.ManagedCursor$IndividualDeletedEntries)
@bci=38, line=470 (Interpreted frame)
    - org.apache.pulsar.broker.service.persistent.PersistentTopic.isOldestMessageExpired(org.apache.bookkeeper.mledger.ManagedCursor,
long) @bci=14, line=1646 (Interpreted frame)
    - org.apache.pulsar.broker.service.persistent.PersistentReplicator.expireMessages(int)
@bci=40, line=571 (Compiled frame)
    - org.apache.pulsar.broker.service.persistent.PersistentTopic.lambda$32(org.apache.pulsar.common.policies.data.Policies,
java.lang.String, org.apache.pulsar.broker.service.Replicator) @bci=8, line=963 (Compiled
frame)
    - org.apache.pulsar.broker.service.persistent.PersistentTopic$$Lambda$407.accept(java.lang.Object,
java.lang.Object) @bci=12 (Compiled frame)
    - org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(java.util.function.BiConsumer)
@bci=121, line=386 (Compiled frame)
    - org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(java.util.function.BiConsumer)
@bci=26, line=160 (Compiled frame)
    - org.apache.pulsar.broker.service.persistent.PersistentTopic.checkMessageExpiry() @bci=86,
line=963 (Compiled frame)
    - org.apache.pulsar.broker.service.BrokerService$$Lambda$366.accept(java.lang.Object)
@bci=4 (Compiled frame)
    - org.apache.pulsar.broker.service.BrokerService.lambda$20(java.util.function.Consumer,
java.lang.String, java.util.concurrent.CompletableFuture) @bci=20, line=887 (Compiled frame)
    - org.apache.pulsar.broker.service.BrokerService$$Lambda$349.accept(java.lang.Object,
java.lang.Object) @bci=12 (Compiled frame)
    - org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(java.util.function.BiConsumer)
@bci=121, line=386 (Compiled frame)
    - org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(java.util.function.BiConsumer)
@bci=26, line=160 (Compiled frame)
    - org.apache.pulsar.broker.service.BrokerService.forEachTopic(java.util.function.Consumer)
@bci=10, line=884 (Compiled frame)
    - org.apache.pulsar.broker.service.BrokerService.checkMessageExpiry() @bci=6, line=861
(Interpreted frame)
    - org.apache.pulsar.broker.service.BrokerService$$Lambda$135.run() @bci=4 (Interpreted
frame)
    - org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun() @bci=4, line=32 (Compiled frame)
    - org.apache.bookkeeper.common.util.SafeRunnable.run() @bci=1, line=36 (Compiled frame)
    - java.util.concurrent.Executors$RunnableAdapter.call() @bci=4, line=511 (Compiled frame)
    - java.util.concurrent.FutureTask.runAndReset() @bci=47, line=308 (Compiled frame)
    - java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
@bci=1, line=180 (Compiled frame)
    - java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run() @bci=37,
line=294 (Compiled frame)
    - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
@bci=95, line=1142 (Compiled frame)
    - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame)
    - io.netty.util.concurrent.FastThreadLocalRunnable.run() @bci=4, line=30 (Interpreted
frame)
    - java.lang.Thread.run() @bci=11, line=748 (Interpreted frame)
   ```
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message