logging-log4net-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mike Baker" <mike.ba...@harmonicinc.com>
Subject RE: Multi-app centralized logging vs. per app logging and background process consolidation
Date Fri, 08 Feb 2008 22:46:06 GMT
Thanks, Dean, for your ideas.  Unfortunately, one of the constraints on
our logging solution is to minimize additional network traffic.
OutputDebugString( ) is an elegant and efficient single-pc solution, as
it uses shared memory (see
http://unixwiz.net/techtips/outputdebugstring.html).

 

I am not sure I agree that the RollingFileAppender approach is so bad.
Typically the logs are write-only (nobody has need to look at them), and
if they do have need examination, there is a specific time window of
interest (just before/after some anomaly).  An alternative to having a
background process to merging is to instead do it on-demand.  This
approach would involving writing a simple merge app with GUI that allows
the user to specify a time range (and perhaps select which log files) of
interest to merge.  

 

The other two goals of our logging solution cause minimal performance
hit when logging and is reliable. 

 

Any other thoughts on this?

 

Thanks again,

-Mike

 

 

 

________________________________

From: Dean Fiala [mailto:dfiala@celadonlabs.com] 
Sent: Thursday, February 07, 2008 9:44 PM
To: log4net-user@logging.apache.org
Subject: re: Multi-app centralized logging vs. per app logging and
background process consolidation

 

The RollingFile merge idea sounds like a lot of extra work and overhead
for little obvious gain.

The Publish-Subscribe sounds like a much better direction, but if you
are putting everything in a central repository, then there is only one
subscriber and there is no need for the overhead of who wants what.
Just publish everything to the central place.  

A couple of 2x4 solutions that require little more than configuration...
1) AdoNetAppenders to to log everything to a centralized database 
2) SmtpAppenders to log everything to a single email account

The advantage of the database is everything is neatly stored in typed
columns, timestamped, and easily sortable and analyzed. Triggers and
alerts will let you do some fancy things if needed. Lots of built in
tools for back-ups, archiving, etc. The disadvantage is that it might
not work with your database/server. The application logging load might
overwhelm the database. Probably not the best answer if the systems are
widely distributed (ie not on the same LAN). You'll probably also want
to build an app to look at the log info in, so there will be some build
time/cost. 

The Smtp option isn't as flexible, but gosh it's cheap and easy and
works not matter how widely flung the servers are.  Throw Outlook (or
Outlook Express) on a box and your have a centralized logging client,
with a modicum of search and sorting capability already built in.  No UI
to build. Heck you could even just use a web mail client. Of course, if
you need to send each log event a single message you might make your
mail server (or administrator) angry, and parsing through the data will
require an export and another app.  You might also have delivery issues
that throw off time synching. 

Another 2x4 solution is to use LocalSysLogAppender/RemoteSyslogAppender
to ultimately get the messages to a single server's syslog service.
There are some free/cheap client tools for working with the syslog
service.  Everything comes in a known log format. There might be some
voodoo involved in getting the syslog services all talking together
and/or using the RemoteSyslogAppender.

If you need to get fancier, an approach based on the RemotingAppender
could work.  You'd need to build a Remoting server application (hosted
as a Windows Service or on  IIS), and a client app to query it.  If
necessary you'll also need to figure out how to persist the log
information and build the appropriate goodies to handle all that.  This
would work on a locally or widely distributed set of servers.  One
possible issue is that buffering could throw off the time synching .  

If you have the infrastructure, you might want to check out the
MSMQAppender.  Would give you very robust message delivery. You'd have
to build similar helper apps as you would in the Remoting solution.

Fun problem to think about.  Good luck.

Regards,

Dean Fiala

________________________________



A previous generation application suite simply uses OutputDebugString( )
to achieve centralized logging for a multi-app suite running on a single
host. The next generation replacement is in the design stage, and will
be distributed. It is desired to maintain an efficient and reliable
means of centralized logging.

One idea is some sort of publish-subscribe model, whereby each app
writes to a publisher, with the publisher responsible for delivering the
messages to each subscriber.

Another idea is to have each app use a RollingFileAppender, and have a
background process examine the time tags to assemble a centralized log
from the individual files. Obviously time-syncing will be necessary
between hosts, and a means to determine any offsets. 

Both of these approaches may involve more work than we would like to
budget for, and may have inherent time-order ambiguity for messages that
are close together due to OS context switching. 

Any comments on these approaches? Any other ideas on how better to do
this?

Thanks,
-Mike





Mime
View raw message