hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Scott Carey (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6884) Add LOG.isDebugEnabled() guard for each LOG.debug("...")
Date Tue, 17 Aug 2010 17:38:17 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899484#action_12899484

Scott Carey commented on HADOOP-6884:

bq. Thanks Luke for clarfying aspectj.

If aspectj can't do this, then its likely a no-go.  Spring or Guice cannot do it either (they
both change methods by overriding them, not wrapping them).
That leaves ASM or low level bytecode changes which definitely will work but would not be
easy at all and would require significant testing.  At this point, a feature like that should
be part of the logging framework.

bq. If I understand the slf4j API correctly, slf4j does not really help for the codes above.
It has to create Integer, Long and Double objects for the primitive data type and an array
for vararg.

It helps a lot, even with the autoboxing and varargs.  A String has a hashCode and byte array
member variable, a minimum memory overhead much larger than a boxed object.
The autoboxing and varargs is significantly less expensive than string concatenation.

int a = 0; long b = 1L; double c = 0.0d;
LOG.debug("a={}, b={}, c={}", a, b, c);

In slf4j this requires allocating a varargs array with 3 elements (~32 bytes), and one Integer,
one Long, and one Double (16, 24, and 24 bytes).
With log4j, it does not require the vararg array, but creates a string out of each number
-- which uses CPU and allocates more memory than just the boxed object, and then concatenates
that into another String.  The end result is at minimum 15 characters (30 bytes) + String
overhead (~48 bytes), and the intermediate result overhead is larger.  If any of the numbers
are not so small, the overhead grows quickly.

The JVM is getting smarter with its Escape Analysis optimizations too -- which will either
eliminate many autobox and vararg object allocations or push them on the stack.  I think Sun's
JVM will already avoid the varargs Object[] construction, and the Double, but not the Long
or Integer, if +UseEscapeAnalysis is on.  That flag becomes the default soon.  

> Add LOG.isDebugEnabled() guard for each LOG.debug("...")
> --------------------------------------------------------
>                 Key: HADOOP-6884
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6884
>             Project: Hadoop Common
>          Issue Type: Improvement
>    Affects Versions: 0.22.0
>            Reporter: Erik Steffl
>            Assignee: Erik Steffl
>             Fix For: 0.22.0
>         Attachments: HADOOP-6884-0.22-1.patch, HADOOP-6884-0.22.patch
> Each LOG.debug("...") should be executed only if LOG.isDebugEnabled() is true, in some
cases it's expensive to construct the string that is being printed to log. It's much easier
to always use LOG.isDebugEnabled() because it's easier to check (rather than in each case
reason whether it's necessary or not).

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message