stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeffrey Nguyen (jeffrngu)" <jeffr...@cisco.com>
Subject Re: Carbon log flooded causing disk full and reboot
Date Wed, 02 Jul 2014 19:00:36 GMT

Thanks Imesh, Isuru, Amila and Martin.

As Martin mentioned, we had log enabled at DEBUG level.  This change will additional level
of protection besides having good logging instrumentation in the code.   With the DailyRollingFileAppender,
we can only hope that log file is rotated every 24 hr before the disk is filled up.

I've committed this change to both master and 4.0.0.  Please feel free to revert or make adjustment
to the size/number of files if you think necessary.

Regards,
-Jeffrey

From: Imesh Gunaratne <imesh@apache.org<mailto:imesh@apache.org>>
Reply-To: "dev@stratos.apache.org<mailto:dev@stratos.apache.org>" <dev@stratos.apache.org<mailto:dev@stratos.apache.org>>
Date: Sunday, June 29, 2014 1:45 PM
To: dev <dev@stratos.apache.org<mailto:dev@stratos.apache.org>>
Subject: Re: Carbon log flooded causing disk full and reboot

Hi Jeffrey,

A great finding! Did this occur while running Stratos in single JVM mode without having any
DEBUG level logs?
Please send these changes to both master and 4.0.0 branches.

Thanks


On Fri, Jun 27, 2014 at 7:52 PM, Jeffrey Nguyen (jeffrngu) <jeffrngu@cisco.com<mailto:jeffrngu@cisco.com>>
wrote:
Hi,

In our Stratos deployment, we experienced issue where wso2carbon.log file grew as much as
13 GB causing disk full and instance reboot and Stratos inaccessible.   We're deploying Stratos
using default profile.   To fix this issue, we modified log4j.properties with the changes
listed below.

Please let me know if it's okay to push this changes upstream.   If so, which branch(es) should
the changes be checked in?

Thanks,
-Jeffrey


diff --git a/products/stratos/modules/distribution/src/main/conf/log4j.properties b/products/stratos/modules/distribution/src
index a85d346..bb37679 100644
--- a/products/stratos/modules/distribution/src/main/conf/log4j.properties
+++ b/products/stratos/modules/distribution/src/main/conf/log4j.properties
@@ -113,7 +113,9 @@ log4j.appender.CARBON_MEMORY.threshold=DEBUG


 # CARBON_LOGFILE is set to be a DailyRollingFileAppender using a PatternLayout.
-log4j.appender.CARBON_LOGFILE=org.apache.log4j.DailyRollingFileAppender
+log4j.appender.CARBON_LOGFILE=org.apache.log4j.RollingFileAppender
+log4j.appender.CARBON_LOGFILE.MaxFileSize=10000KB
+log4j.appender.CARBON_LOGFILE.MaxBackupIndex=10
 # Log file will be overridden by the configuration setting in the DB
 # This path should be relative to WSO2 Carbon Home



--
Imesh Gunaratne

Technical Lead, WSO2
Committer & PPMC Member, Apache Stratos

Mime
View raw message