Mailing-List: contact log4j-dev-help@logging.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Log4J Developers List" <log4j-dev@logging.apache.org>
Date: Sun, 8 May 2016 14:04:12 +0000 (UTC)
From: "Remko Popma (JIRA)" <jira@apache.org>
To: log4j-dev@logging.apache.org
Message-ID: <JIRA.12844024.1436491051000.137781.1462716252866@Atlassian.JIRA>
In-Reply-To: <JIRA.12844024.1436491051000@Atlassian.JIRA>
References: <JIRA.12844024.1436491051000@Atlassian.JIRA> <JIRA.12844024.1436491051681@arcas>
Subject: [jira] [Commented] (LOG4J2-1080) Drop events when the RingBuffer is
 full
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Sun, 08 May 2016 14:04:14 -0000


    [ https://issues.apache.org/jira/browse/LOG4J2-1080?page=3Dcom.atlassia=
n.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D152=
75601#comment-15275601 ]=20

Remko Popma commented on LOG4J2-1080:
-------------------------------------

I renamed {{AsyncEventRouter}} to {{AsyncQueueFullPolicy}},
{{AsyncEventRouterFactory}} to {{AsyncQueueFullPolicyFactory}},
and the system property is also now called {{log4j2.AsyncQueueFullPolicy}}.

Initially the plan was to allow discarding events even when the queue was p=
artially full, so I chose the more general name AsyncEventRouter. Now that =
the logic has been changed so this policy only applies when the queue is ac=
tually full, I think a more specific (and more descriptive) name is better.


> Drop events when the RingBuffer is full
> ---------------------------------------
>
>                 Key: LOG4J2-1080
>                 URL: https://issues.apache.org/jira/browse/LOG4J2-1080
>             Project: Log4j 2
>          Issue Type: New Feature
>            Reporter: tzachi
>            Assignee: Remko Popma
>             Fix For: 2.6
>
>         Attachments: AsyncLogger.dropEvents.patch
>
>
> I am running into performance issue with an appender, in a certain scenar=
io (attached at the bottom), that causes RingBuffer to reach its full capac=
ity. When that happens I can see that my app throughput drops significantly=
.
> I think it will be really useful to be able to configure the RingBuffer h=
andler to be able to drop events whenever the buffer reaches its capacity, =
instead of what seems currently as blocking, as I don't want the logging to=
 affect the main application.
> ---------------------------------------------------------------------
> Here is the scenario that led me to this request:
> I am currently testing the log4j-flume-ng appender and running into some =
issues. It seems like whenever log4j appender fails to log an event it caus=
es the disruptor ring buffer to get full which slows down the whole system.
> My setup looks more or less like that:=20
> process 1: Java app which uses log4j2 (with flume-ng=E2=80=99s Avro appen=
der)
> process 2: local flume-ng which gets the logs on using an Avro source and=
 process them=20
> Here are my findings:
> When Flume (process 2) is up and running, everything actually looks reall=
y good. The ring buffer capacity is almost always full and there are no per=
formance issues. The problem starts when I shut down process 2 - I am tryin=
g to simulate a case in which this process crashes, as I do not want it to =
effect process 1. As soon as I shut down flume I start getting exceptions p=
roduced by log4j telling me they cannot append the log - so far it makes se=
nse. The thing is, that at the same time I can see that the ring buffer sta=
rts to fill up. As long as it=E2=80=99s not totally full process=E2=80=99s =
1 throughput stays the same. The problem gets serious as soon as the buffer=
 reaches full capacity. When that happens the throughput drops in 80% and i=
t does not seem to recover from this state. But, as soon as I restart proce=
ss 2, things get back to normal pretty quick - the buffer gets emptied, and=
 the throughput climbs back to what it was before. I assume that from some =
reason a fail to append makes the RingBuffer consumer thread significantly =
slower.
> Besides checking why the flume appender preform slower when an exception =
is thrown, I wish I could just discard the log events when the buffer gets =
full.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org