logging-log4net-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Blake Bishop" <bbis...@sorenson.com>
Subject Re: High occurrence errors
Date Thu, 03 Nov 2005 23:20:35 GMT
Ron,

The concepts behind your solution to the throttling problem would make a 
great addition to the log4net documentation IMHO, possibly under some new 
documentation topic/subtopic like "Filter Examples/Logging Every Nth 
Message" or something like that.

Cheers,
Blake

"Ron Grabowski" <rongrabowski@yahoo.com> wrote in message 
news:20051102200616.93984.qmail@web32209.mail.mud.yahoo.com...
> This will generate a lot of messages with exceptions then generate a
> few exceptions more slowly:
>
> for (int i=0; i < 10000; i++)
> {
>  log.Debug(
>   "ApplicationException " + i,
>   new ApplicationException("ApplicationException " + i));
> }
> System.Threading.Thread.Sleep(5000); // 5 seconds
> log.Debug(
>  "ApplicationException " + DateTime.Now,
>  new ApplicationException("ApplicationException " + DateTime.Now));
> System.Threading.Thread.Sleep(5000); // 5 seconds
> log.Debug(
>  "ApplicationException " + DateTime.Now,
>  new ApplicationException("ApplicationException " + DateTime.Now));
> System.Threading.Thread.Sleep(5000); // 5 seconds
> log.Debug(
>  "ApplicationException " + DateTime.Now,
>  new ApplicationException("ApplicationException " + DateTime.Now));
>
> It sounds like you only want to process the logging event if the
> exception occured more than a specified interval from the last
> exception of the same type? This output is from a filter that accepts a
> logging event only if the logging event was at least 2 seconds from the
> previous logging event with the same exception:
>
> DEBUG 2005-11-02 03:07:26 PM - ApplicationException 0
> System.ApplicationException: ApplicationException 0
> DEBUG 2005-11-02 03:07:28 PM - ApplicationException 2555
> System.ApplicationException: ApplicationException 2555
> DEBUG 2005-11-02 03:07:30 PM - ApplicationException 5440
> System.ApplicationException: ApplicationException 5440
> DEBUG 2005-11-02 03:07:32 PM - ApplicationException 8333
> System.ApplicationException: ApplicationException 8333
> DEBUG 2005-11-02 03:07:38 PM - ApplicationException 11/2/2005 3:07:38
> PM
> System.ApplicationException: ApplicationException 11/2/2005 3:07:38 PM
> DEBUG 2005-11-02 03:07:43 PM - ApplicationException 11/2/2005 3:07:43
> PM
> System.ApplicationException: ApplicationException 11/2/2005 3:07:43 PM
> DEBUG 2005-11-02 03:07:48 PM - ApplicationException 11/2/2005 3:07:48
> PM
> System.ApplicationException: ApplicationException 11/2/2005 3:07:48 PM
>
> Threshold probably isn't the best name for the internval property:
>
> <filter type="Company.Project.Logging.ExceptionThrottleFilter">
>  <threshold value="2" />
>  <exceptionType value="System.ApplicationException" />
> </filter>
>
> See the attached class for how I implemented it. You should be able to
> chain the filters too. A faster implementation may be to subtract Ticks
> instead of using DateTime's Subtract method.
>
> Another approach would be to change the ExceptionObject to just the
> string representation of its type:
>
> loggingEvent.ExceptionObject = loggingEventObject.GetType().ToString()
>
> if some criteria is met. That would allow you to keep the exceptions
> while reducing the size of the log files. This could be done in as a
> Layout or a Converter.
>
> - Ron
>
> --- Russell Haley <rhaley@axys.com> wrote:
>
>> Hello again to the log4net group!
>>
>> I am currently using L4N to log all the errors in a real time data
>> acquisition application and have come across a bit of a problem.
>> There are
>> certain places in the application where exceptions occur at a very
>> high rate
>> when a problem does arise and the log files generated become huge in
>> a
>> matter of minutes (or in one case, seconds). The specific places that
>> I have
>> found this to be problematic is in a serial driver and in the class
>> that
>> stores the data to a database.
>>
>> Although L4N can limit the size of the files and the number of "roll
>> overs",
>> I (read "My Boss") would also like to limit the rate at which these
>> exceptions are actually logged. Is there any built in mechanism that
>> I can
>> use to achieve this? I would still like to know that the errors are
>> occurring, but in the cases that I am thinking of, I don't need to
>> know that
>> one or two exceptions occurred 35 times in one second! I am looking
>> at
>> implementing some sort of "buffer" that holds the past x errors and
>> has a
>> threshold that defines the time period between exceptions logged, but
>> again,
>> I was wondering if there was anything internal to L4N to achieve
>> this?
>>
>> I'm well aware that I need to eliminate the problematic exceptions,
>> but I
>> still need to have some mechanism to find out when they occur. In the
>> case
>> of the serial driver, it has been running very well but the other day
>> I
>> tried using a Serial to USB dongle (converter) that caused problems.
>> I have
>> since eliminated that specific problem, but as I'm sure everyone is
>> aware,
>> these things are not always foreseeable.
>>
>> Any suggestions would be greatly appreciated. Thank you once again
>> for your
>> help.
>>
>> Sincerely,
>>
>> Russell Haley
>>
>>


--------------------------------------------------------------------------------


> public class ExceptionThrottleFilter : FilterSkeleton
> {
> private DateTime lastException = DateTime.MinValue;
> private Type exceptionType = typeof(Exception);
> private int threshold = 5; // seconds
>
> public override void ActivateOptions()
> {
> base.ActivateOptions();
> }
>
> public override FilterDecision Decide(LoggingEvent loggingEvent)
> {
> if (loggingEvent.ExceptionObject != null && 
> loggingEvent.ExceptionObject.GetType() == exceptionType)
> {
> if (loggingEvent.TimeStamp.Subtract(lastException).TotalSeconds > 
> threshold)
> {
> lastException = loggingEvent.TimeStamp;
> return FilterDecision.Accept;
> }
> else
> {
> return FilterDecision.Deny;
> }
> }
> else
> {
> return FilterDecision.Neutral;
> }
> }
>
> public Type ExceptionType
> {
> get { return exceptionType; }
> set { exceptionType = value; }
> }
>
> public int Threshold
> {
> get { return threshold; }
> set { threshold = value; }
> }
> } 




Mime
View raw message