hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Colin McCabe <cmcc...@alumni.cmu.edu>
Subject Re: [DISCUSS] Switch to log4j 2
Date Mon, 18 Aug 2014 20:32:05 GMT
On Fri, Aug 15, 2014 at 8:50 AM, Aaron T. Myers <atm@cloudera.com> wrote:
> Not necessarily opposed to switching logging frameworks, but I believe we
> can actually support async logging with today's logging system if we wanted
> to, e.g. as was done for the HDFS audit logger in this JIRA:
>
> https://issues.apache.org/jira/browse/HDFS-5241

Yes, this is a great example of making something async without
switching logging frameworks.  +1 for doing that where it is
appropriate.

>
> --
> Aaron T. Myers
> Software Engineer, Cloudera
>
>
> On Fri, Aug 15, 2014 at 5:44 AM, Steve Loughran <stevel@hortonworks.com>
> wrote:
>
>> moving to SLF4J as an API is independent —it's just a better API for
>> logging than commons-logging, was already a dependency and doesn't force
>> anyone to switch to a new log back end.

Interesting idea.  Did anyone do a performance comparison and/or API
comparison with SLF4j on Hadoop?

>>
>>
>> On 15 August 2014 03:34, Tsuyoshi OZAWA <ozawa.tsuyoshi@gmail.com> wrote:
>>
>> > Hi,
>> >
>> > Steve has started discussion titled "use SLF4J APIs in new modules?"
>> > as a related topic.
>> >
>> >
>> http://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201404.mbox/%3CCA+4kjVv_9cMmTDQZCgZy-CHSLyB1WkGdUNXS7wrhesLwbuHJjQ@mail.gmail.com%3E
>> >
>> > It sounds good to me to use asynchronous logging when we log INFO. One

-1.  Async logging for everything will make a lot of failures
un-debuggable.  Just to give one example, what if you get a JVM out of
memory crash?  You'll lose the last few log messages which could have
told you what was going on.  Even if the JVM doesn't terminate, log
messages will be out of order, which is annoying, and will make
debugging harder.

The kernel already buffers the log files in memory.  Not every log
message generates a disk seek.  But on the other hand, if the JVM
process crashes, you've got everything.  In other words, we've already
got as much buffering and asynchronicity as we need!

If the problem is that the noisy logs are overloading the disk
bandwidth, that problem can't be solved by adding Java-level async.
You need more bandwidth.  A simple way of doing this is putting the
log partition on /dev/shm.  We could also look into stripping some of
the boilerplate from log messages-- there are a lot of super-long log
messages that could be much more concise.  Other Java logging
frameworks might have less overhead (I'm not an expert on this, but
maybe someone could post some numbers?)

best,
Colin


>> > concern is that asynchronous logging makes debugging difficult - I
>> > don't know log4j 2 well, but I suspect that ordering of logging can be
>> > changed even if WARN or  FATAL are logged with synchronous logger.
>> >
>> > Thanks,
>> > - Tsuyoshi
>> >
>> > On Thu, Aug 14, 2014 at 6:44 AM, Arpit Agarwal <aagarwal@hortonworks.com
>> >
>> > wrote:
>> > > I don't recall whether this was discussed before.
>> > >
>> > > I often find our INFO logging to be too sparse for useful diagnosis. A
>> > high
>> > > performance logging framework will encourage us to log more.
>> > Specifically,
>> > > Asynchronous Loggers look interesting.
>> > > https://logging.apache.org/log4j/2.x/manual/async.html#Performance
>> > >
>> > > What does the community think of switching to log4j 2 in a Hadoop 2.x
>> > > release?
>> > >
>> > > --
>> > > CONFIDENTIALITY NOTICE
>> > > NOTICE: This message is intended for the use of the individual or
>> entity
>> > to
>> > > which it is addressed and may contain information that is confidential,
>> > > privileged and exempt from disclosure under applicable law. If the
>> reader
>> > > of this message is not the intended recipient, you are hereby notified
>> > that
>> > > any printing, copying, dissemination, distribution, disclosure or
>> > > forwarding of this communication is strictly prohibited. If you have
>> > > received this communication in error, please contact the sender
>> > immediately
>> > > and delete it from your system. Thank You.
>> >
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>

Mime
View raw message