logging-log4j-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Barker (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LOG4J2-1221) Dead lock observed in BlockingWaitStrategy in Log 4J
Date Tue, 12 Jan 2016 21:16:39 GMT

    [ https://issues.apache.org/jira/browse/LOG4J2-1221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094961#comment-15094961
] 

Michael Barker commented on LOG4J2-1221:
----------------------------------------

>From the analysis I did on the original Github issue it looks like the Disruptor gets
stuck waiting on a notification from the publisher.  It looks like the signal from the condition
variable is somehow missed.  I'm still trying to dig into the reasons why this occurred. 
I've never seen it on Linux, so it could be a problem with the Solaris build of the JDK, the
Solaris kernel or a bug in the Disruptor that I've missed.  However there is a workaround.
 I have a patch for Log4J that allows for the use of the TimeoutBlockingWaitStrategy, which
will periodically wake up from the signal.await call.  So if a notification is missed somehow
it will recover with a small latency delay (default 10ms).

I've attached the patch to this ticket.

> Dead lock observed in BlockingWaitStrategy in Log 4J
> ----------------------------------------------------
>
>                 Key: LOG4J2-1221
>                 URL: https://issues.apache.org/jira/browse/LOG4J2-1221
>             Project: Log4j 2
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 2.2
>         Environment: log4J Version : 2.2 Disruptor Version : 3.3.2
> Ring Buffer Size : 128
> OS Version :
> cat /etc/release
> Oracle Solaris 11.2 X86
> Java Version
> java version "1.7.0_45"
> Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
> Java HotSpot(TM) Server VM (build 24.45-b08, mixed mode)
>            Reporter: Sampath Kumar
>            Priority: Critical
>              Labels: patch
>             Fix For: 2.5.1
>
>         Attachments: log4j2-test.xml, timeout_wait_strategy.patch
>
>
> We have seen this behavior in during high load. Where Logging Got Stropped and Application
Went to not responsive state.
> log4J Version : 2.2 Disruptor Version : 3.3.2
> Ring Buffer Size : 128
> Producer(Multiples Threads) and Consumer Threads(Single Thread As per Log 4J Configuration)
Started Waiting on each other.
> Here is the one of the Trace from Thread Dump:
> Producer : 
> "[ACTIVE] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)'" TIMED_WAITING
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:349)
> com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:136)
> com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:105)
> com.lmax.disruptor.RingBuffer.publishEvent(RingBuffer.java:444)
> com.lmax.disruptor.dsl.Disruptor.publishEvent(Disruptor.java:256)
> org.apache.logging.log4j.core.async.AsyncLogger.logMessage(AsyncLogger.java:285)
> org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:722)
> org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:693)
> org.apache.logging.log4j.jcl.Log4jLog.debug(Log4jLog.java:81)
> Consumer Thread :
> "AsyncLogger-1" waiting for lock java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5d972983
WAITING
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> com.lmax.disruptor.BlockingWaitStrategy.waitFor(BlockingWaitStrategy.java:45)
> com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:55)
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:123)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> java.lang.Thread.run(Thread.java:744)
> Is this is known issue which got already fixed in recent build ?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org


Mime
View raw message