kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Francois Saint-Jacques (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-1115) producer performance affected by trace/debug calls
Date Mon, 04 Nov 2013 01:12:17 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-1115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13812550#comment-13812550
] 

Francois Saint-Jacques commented on KAFKA-1115:
-----------------------------------------------

You are right, and by looking at the code the trace/debug logging functions are defined with
lazy arguments.

By debugging a bit further, I did not have any log4.properties in the producer classpath.
The library does give me a WARN about missing log4.properties file. After this warning message,
the program does not output any more log. Internally I'd assume the logging still we're set
to TRACE without any appender to stdout/stderr.

I'll take WARN more seriously next time. Maybe a put message in the producer documentation
that not providing log4j configuration will have a serious impact on performance.

On another note, I believe trace/debug message shouldn't be committed to production code,
but this is only a personal hunch.

> producer performance affected by trace/debug calls
> --------------------------------------------------
>
>                 Key: KAFKA-1115
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1115
>             Project: Kafka
>          Issue Type: Improvement
>          Components: producer 
>    Affects Versions: 0.8
>            Reporter: Francois Saint-Jacques
>            Assignee: Francois Saint-Jacques
>         Attachments: producer-performance-fix.patch
>
>
> After investigating high CPU usage on some producers in production, we found out that
a lot of time was passed in string construction for logging of DEBUG and TRACE level.
> This patch removes most of the logging calls, on our systems it cuts CPU usage down to
half of what it used to be.
> Note that this is a significant boost in performance for environment where there are
thousands of msg/s.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message