Hi,

A recent discussion on IRC and the mailing list, followed by the creation of DIRSERVER-1871 [1] made me look at the way logs could be activated and specifically the 'log4j.properties' file we bundle in the distributions.

Here's the content of the current version of this file:
#############################################################################
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#############################################################################
log4j.rootCategory=WARN, R, stdout

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout

log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=${apacheds.log.dir}/apacheds.log

log4j.appender.R.MaxFileSize=1024KB
# Keep some backup files
log4j.appender.R.MaxBackupIndex=5

log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=[%d{HH:mm:ss}] %p [%c] - %m%n

log4j.appender.stdout.layout.ConversionPattern=[%d{HH:mm:ss}] %p [%c] - %m%n

# with these we'll not get innundated when switching to DEBUG
log4j.logger.jdbm.recman.BaseRecordManager=FATAL
log4j.logger.org.apache.directory.server.OPERATION_LOG=FATAL
log4j.logger.org.apache.directory.server.OPERATION_TIME=FATAL
log4j.logger.org.apache.directory.api.CODEC_LOG=FATAL
log4j.logger.org.apache.directory.server.ACI_LOG=FATAL
log4j.logger.org.apache.directory.server.PROVIDER_LOG=FATAL
log4j.logger.org.apache.directory.server.CONSUMER_LOG=FATAL
log4j.logger.org.apache.directory.CURSOR_LOG=FATAL
log4j.logger.org.apache.directory.api=FATAL
log4j.logger.org.apache.directory.server=FATAL
log4j.logger.net.sf.ehcache=FATAL
log4j.logger.org.apache.mina=FATAL

A few comments on it.

First, we have two appenders ('R' & 'stdout').
I don't really see the utility of having these two. The RollingFileAppender should be enough.

What happens now with the ConsoleAppender is that any log we can find in the 'apacheds.log' is then duplicated in the 'wrapper.log' file, mandatory log file for the Tanuki wrapper whichs logs succesful or failed launched and termination attempts of the server.

Personally, I would get rid of the ConsoleAppender (unfortunately, we will have two different log files but there's not that can be done about that).

I also took the liberty to modify the RollingFileAppender settings to something more useful than the current values.
I increased 'MaxFileSize' to '100MB' and decreased 'MaxBackupIndex' to '3'.
So, at most that would be 300 MB dedicated to logs.
It looks enough for me.

Last but not least, I fixed what was causing most of our log issues.
At the end of the file, there are a lot of specific loggers all set to FATAL.
Two are especially interesting, 'org.apache.directory.api' & 'org.apache.directory.server'. Setting these ones to 'FATAL' means that we will only log severe errors for all classes of the LDAP API and ApacheDS.
But it also means that the setting of the root logger at the beginning of the file has absolutely no effect, hence the problems we've been having with log activations.

I propose that we keep these definitions, which are very useful when we need to debug things in the server, but that they should be set with a 'DEBUG'  log level and be deactivated by default.

In the end, an updated version of the file looks like this:
#############################################################################
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#############################################################################
log4j.rootCategory=WARN, F

log4j.appender.F=org.apache.log4j.RollingFileAppender
log4j.appender.F.File=${apacheds.log.dir}/apacheds.log
log4j.appender.F.MaxFileSize=100MB
log4j.appender.F.MaxBackupIndex=3
log4j.appender.F.layout=org.apache.log4j.PatternLayout
log4j.appender.F.layout.ConversionPattern=[%d{HH:mm:ss}] %p [%c] - %m%n

# Specific loggers (useful for debugging)
#log4j.logger.jdbm.recman.BaseRecordManager=DEBUG
#log4j.logger.org.apache.directory.server.OPERATION_LOG=DEBUG
#log4j.logger.org.apache.directory.server.OPERATION_TIME=DEBUG
#log4j.logger.org.apache.directory.api.CODEC_LOG=DEBUG
#log4j.logger.org.apache.directory.server.ACI_LOG=DEBUG
#log4j.logger.org.apache.directory.server.PROVIDER_LOG=DEBUG
#log4j.logger.org.apache.directory.server.CONSUMER_LOG=DEBUG
#log4j.logger.org.apache.directory.CURSOR_LOG=DEBUG
#log4j.logger.org.apache.directory.api=DEBUG
#log4j.logger.org.apache.directory.server=DEBUG
#log4j.logger.net.sf.ehcache=DEBUG
#log4j.logger.org.apache.mina=DEBUG


I also added the documentation related to logs in the Advanced User Guide, reflecting my 'log4j.properties' proposal, which is accessible in the staging website:
http://directory.staging.apache.org/apacheds/advanced-ug/5.3-logs.html

WDYT?
Reviews welcomed!

Regards,
Pierre-Arnaud


[1] - https://issues.apache.org/jira/browse/DIRSERVER-1871