Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@kafka.apache.org
Date: Fri, 1 Nov 2013 23:34:18 +0000 (UTC)
From: "Francois Saint-Jacques (JIRA)" <jira@apache.org>
To: dev@kafka.apache.org
Message-ID: <JIRA.12677162.1383340827445.9390.1383348858115@arcas>
In-Reply-To: <JIRA.12677162.1383340827445@arcas>
References: <JIRA.12677162.1383340827445@arcas>
Subject: [jira] [Commented] (KAFKA-1115) producer performance affected by
 trace/debug calls
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/KAFKA-1115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13811759#comment-13811759 ] 

Francois Saint-Jacques commented on KAFKA-1115:
-----------------------------------------------

This is not the problem. Even if I change the log level, the construction of the string passed to the logging subsystem is the root cause, i.e. all calls of the form "Message %s is... ".format(...) . Scala is not a lazy evaluation language, the only ways to fix this problem are:

1. Wrap all the trace/debug calls with if (logging.debugEnabled()...) so that it doesn't get evaluated. This is the lazy way.

or

2. Remove any debug/trace calls in this critical code path. This is clearly the remnant of `print foo' debugging, it shouldn't be committed to the trunk branch. 

> producer performance affected by trace/debug calls
> --------------------------------------------------
>
>                 Key: KAFKA-1115
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1115
>             Project: Kafka
>          Issue Type: Improvement
>          Components: producer 
>    Affects Versions: 0.8
>            Reporter: Francois Saint-Jacques
>            Assignee: Francois Saint-Jacques
>         Attachments: producer-performance-fix.patch
>
>
> After investigating high CPU usage on some producers in production, we found out that a lot of time was passed in string construction for logging of DEBUG and TRACE level.
> This patch removes most of the logging calls, on our systems it cuts CPU usage down to half of what it used to be.
> Note that this is a significant boost in performance for environment where there are thousands of msg/s.


--
This message was sent by Atlassian JIRA
(v6.1#6144)