logging-log4j-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ceki Gülcü <c...@qos.ch>
Subject RE: Gzip rolled logs
Date Mon, 02 Aug 2004 13:55:25 GMT
At 12:01 PM 8/2/2004, you wrote:
>Ceki Gülcü wrote:
>
> >
> > Hello Davor,
> >
> > The o.a.l.rolling package (available from CVS) already contains an
> > implementation of zipped and rolled log files.
>
>Yes, I just realized that :-)
> >
> > Paul: It really already there! :-)
> >
> > The existing RollingFileAppender and ilk are deprecated. This means
> > that they are being phased out and will not be enhanced except for bug
> > fixes.
> >
> > I propose that you have a look at the current implementation and
> > compare it with what you have. The criticism by a knowledgeable party
> > is always highly welcome.
>
>Well, after a short glance at the codebase, I'd just suggest a threaded
>Compress implementation.
>
>While I was developing the before mentioned appenders, gzip compression used
>to take about 85 seconds to compress a 200mb file. If you have a
>minute-based rolling appender and collect about those 200mb in one minute,
>then strange things could happen (log4j does output buffering, obviously,
>but while the compression takes place, no logs are visible etc etc...)

The fact that logging will be blocked while compressing large files is a 
very valid point. My suggestion would be to use the log4j scheduler to take 
care of the compression. Alternatively, one could launch a separate thread 
to handle the compression.

>Anyway, threaded deflater solved the buffering problem, but revealed
>OutOfMemoryError one: while one file is being compressed, another thread is
>just about to start, which eventually could lead to OOME. Some thread pool
>might be a good choice, with parameters such as max concurrent compress
>threads or maybe max size for parallel compression (in megabytes).

Why is there a need for more than 1 compression thread? I don't follow you.

>I'll take a detailed look a little bit later.
>
>P.S. Some not-so-obvious libraries are needed to successfully compile the
>newest CVS HEAD code. Are they available from one place or maybe I need to
>hunt them somewhere in the net jungle?

As long as you use Ant to compile log4j, it is easy. Our Ant build script 
will skip the compilation of components with missing dependencies. The 
problem arises when the user tries to compile with an IDE such as Eclipse. 
This is an important concern requiring further investigation.


>--
>Davor Cengija, dcengija_IQ_Filter_@inet.hr

-- 
Ceki Gülcü

      For log4j documentation consider "The complete log4j manual"
      ISBN: 2970036908 http://www.qos.ch/shop/products/clm_t.jsp  



---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org


Mime
View raw message