logging-log4net-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dongsheng Song (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (LOG4NET-415) RemoteSyslogAppender may block for ARP resolution + Improvement Strict RFC3164
Date Thu, 09 Jan 2014 08:35:50 GMT

    [ https://issues.apache.org/jira/browse/LOG4NET-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13865013#comment-13865013
] 

Dongsheng Song edited comment on LOG4NET-415 at 1/9/14 8:35 AM:
----------------------------------------------------------------

Just Set Socket.Blocking to false is not correct non-block programing,
this will cause huge unexpected message lost.

Here is a demo:

{quote}
Message lost rate: 32.61 %,  32.61 %
Message lost rate: 32.90 %,  32.76 %
Message lost rate: 27.28 %,  30.81 %
Message lost rate: 55.46 %,  36.56 %
...
{quote}

Here is my test code:

{code:borderStyle=solid}
    class MessageLostTest
    {
        private static long _messageCount;
        private static long _errorCount;

        private static void Main()
        {
            Socket socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
            socket.Connect("192.168.30.233", 514);
            socket.Blocking = false;

            new Thread(delegate(object obj)
            {
                long m = _messageCount, e = _errorCount;
                while (true)
                {
                    Thread.Sleep(5000);
                    long m2 = Interlocked.Read(ref _messageCount);
                    long e2 = Interlocked.Read(ref _errorCount);
                    Console.WriteLine("Message lost rate: {0:##.00 %}, {1: ##.00 %}",
                                      (e2 - e) / (double)(m2 - m), e2 / (double)m2);
                    m = m2; e = e2;
                }
            }).Start();

            byte[] message = new byte[480];
            while (true)
            {
                try
                {
                    Interlocked.Increment(ref _messageCount);

                    if (socket.Send(message) != message.Length)
                        Interlocked.Increment(ref _errorCount);
                }
                catch (Exception e)
                {
                    Interlocked.Increment(ref _errorCount);
                    //Console.WriteLine("Socket.Send failed: {0}{1}{2}", e.Message, Environment.NewLine,
e.StackTrace);
                }
            }
        }
    }
{code}


was (Author: dongsheng):
Just Set Socket.Blocking to false is not correct non-block programing,
this will cause huge unexpected message lost.

Here is a demo:

Message lost rate: 32.61 %,  32.61 %
Message lost rate: 32.90 %,  32.76 %
Message lost rate: 27.28 %,  30.81 %
Message lost rate: 55.46 %,  36.56 %
...

Here is my test code:

    class MessageLostTest
    {
        private static long _messageCount;
        private static long _errorCount;

        private static void Main()
        {
            Socket socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
            socket.Connect("192.168.30.233", 514);
            socket.Blocking = false;

            new Thread(delegate(object obj)
            {
                long m = _messageCount, e = _errorCount;
                while (true)
                {
                    Thread.Sleep(5000);
                    long m2 = Interlocked.Read(ref _messageCount);
                    long e2 = Interlocked.Read(ref _errorCount);
                    Console.WriteLine("Message lost rate: {0:##.00 %}, {1: ##.00 %}",
                                      (e2 - e) / (double)(m2 - m), e2 / (double)m2);
                    m = m2; e = e2;
                }
            }).Start();

            byte[] message = new byte[400];
            while (true)
            {
                try
                {
                    if (socket.Send(message) != message.Length)
                        Interlocked.Increment(ref _errorCount);

                    Interlocked.Increment(ref _messageCount);
                }
                catch (Exception e)
                {
                    Interlocked.Increment(ref _errorCount);
                    //Console.WriteLine("Socket.Send failed: {0}{1}{2}", e.Message, Environment.NewLine,
e.StackTrace);
                }
            }
        }
    }


> RemoteSyslogAppender may block for ARP resolution + Improvement Strict RFC3164
> ------------------------------------------------------------------------------
>
>                 Key: LOG4NET-415
>                 URL: https://issues.apache.org/jira/browse/LOG4NET-415
>             Project: Log4net
>          Issue Type: Bug
>          Components: Appenders
>    Affects Versions: 1.2.13
>         Environment: Any Windows environment
>            Reporter: Jose Luis Pedrosa
>              Labels: RemoteSyslogAppender
>         Attachments: LOG4NET-415.patch, MessageLostTest.cs, MessageLostTest.cs, MessageLostTestAsync.cs,
RemoteSyslogNonBlockingv2.patch
>
>
> Sending UDP packages may block for some time in specific circumstances:
> 1) Next hop in network level 3 can't be resolved by ARP.
> 2) Datagram size exceeds FastSendDatagramThreshold configured size.
> When sending packets bigger than FastSendDatagramThreshold, the OS waits until the packet
is actually sent, if the If the syslog (or the next hop to reach the syslog) is in the same
VLAN/Subnet the OS tries to resolve by ARP the Ip of the configured syslog, this may take
up to 3 seconds, slowing down the whole application, which in some cases can lead to outages
(timeouts, DB locks...).
> Also the fact that each carriage return generates the headers of the packet again, that
can lead to a significant overhead in some scenarios, for instance when loggign HTTP requests
to a remote syslog, every header will go in a different message. Also some logging may require
characters that are now skipped in patch:  https://issues.apache.org/jira/browse/LOG4NET-370
> I'm adding a patch that
> 1) Moves the use of UDPClient to Non blocking sockets, which eliminates the blocking.

> 2) Adds a configuration field to decide if you want Strict RFC Behaviour or not (with
default Strict).
> Please your feedback, thanks in advance
> Jose Luis



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message