logging-log4j-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pascal Rheaume (QC/EMC)" <Pascal.Rhea...@ericsson.com>
Subject RE: BufferSize always the same
Date Thu, 02 Oct 2003 17:57:35 GMT
Sorry for not being clear.

First, for my main question. Here is an example. My code does a 1000 for loop with log statements
in it. The resulting file is, for example, 1,000 kb. So one loop writes for 1kb. As we said,
8kb is the default buffer value. To have 8kb, we need 8 loops. What will happen here is that
there will be a range of loops (here from 1000 to 1007) that will result in a 1,000 kb log
file. What that means is I would lose, in the worst case, 7.99kb.

What I understand on this lost of logs is it is because the buffer is not full, so it is not
written into the file. What I would like to do is to make the gap of lost data less wide,
let's say 1kb. Is it possible to do that? (If not why would there be the BufferSize parameter...)

What I try to do is to change the value of the BufferSize parameter in my .xml file (see code
below). I think it is suppose to change the default size value to the one I set. But in my
case, it does not. The question is where can it go wrong?

I even try to hardcode the value in the FileAppender.java file: it did not work.

As for the "amazing improvement". In my case, the execution time passes from 15ms without
buffered IO to 2ms with buffered IO. I'm using jdk1.3.1_03. If you say its not supposed to
be that big, I would put it on the OS where my application runs, which is not Windows nor
Unix/Linux. ;)

Thank you!

...
<appender name="FILE" class="MyFileAppender">
		<param name="File" value="/logs/root.log"/>
		<param name="printStackTrace" value="true"/>
		<param name="ImmediateFlush" value="false"/>
		<param name="BufferedIO" value="true"/>
		<param name="BufferSize" value="1024"/>
		<param name="MaxFileSize" value="10MB"/>
		<param name="MaxBackupIndex" value="5"/>
		<layout class="MyLayout">
  		     <param name="ConversionPattern" 
                     value="%-5p: from %C{1} [%-19d{ISO8601}] [%t]  %n      %-40m %n%n"/>
			<param name="Level" value="DEBUG, WARN, ERROR, FATAL" />
		</layout>
		<layout class="MyLayout">
  		     <param name="ConversionPattern" 
                     value="%m%n"/>
			<param name="Level" value="TRACE, INFO" />
		</layout>
	</appender>
...


-----Original Message-----
From: Ceki Gülcü [mailto:ceki@qos.ch]
Sent: Thursday, October 02, 2003 11:16 AM
To: Log4J Users List
Subject: Re: BufferSize always the same



I should have verified for typos before posting my reply. Corrections below.

At 05:10 PM 10/2/2003 +0200, you wrote:
>Well, in Java IO there are buffers and buffers.
>
>Even if you do *not* set the bufferedIO option in FileAppender, that is if 
>bufferedIO is set to false, writing to the file is buffered. 
>FileOutputStream is buffered.

I mean to say that FileOutputStream *always* buffers writes to the hard 
disk independently of FileAppender's bufferedIO option.

>If you set the bufferedIO option, FileAppender will buffer String 
>(unicode) to stream conversions, FileOutputStream will continue to buffer 
>writes to physical media.

FileAppender's bufferedIO option only saves String (unicode) to stream 
conversions.

>At 09:26 AM 10/2/2003 -0500, Pascal Rheaume (QC/EMC) wrote:
>>Hi,
>>
>>I'm using a custom FileAppender with buffered io because speed is really 
>>important for me. I have amazing improvement over not buffered io, so I 
>>would like to keep it buffered. I'm able to live with a small lost of 
>>logging at the end due to the buffer that has not been emptied. And if 
>>I'm correct, the default size for the buffer is 8kb. For me, 8kb is too 
>>big so I want to reduce this size.
>
>Yes, the default buffer for buffered for BufferedWriter is 8KB.

Yes, the default buffer for BufferedWriter (created by FileAppender when 
bufferedIO=yes) is 8KB.

>I am surprised to learn that the bufferedIO option yields in "an amazing 
>improvement". Which JDK version are you using?
>
>
>>Here is my problem: whatever number I put in the BufferSize parameter, it 
>>is ALWAYS a 8kb buffer. The final size of my log file is 8kb or less than 
>>the log file I would have for the same operations sequence in a not 
>>buffered IO.
>
>I fail to not understand the above sentence.
>
>
>>I set the ImmediateFlush parameter to false and the BufferedIO parameter 
>>to true. So, anybody has an idea on what could be the source of my 
>>problem here? I really don' get it!
>>
>>I have a secondary question: that BufferSize parameter, is it a size in 
>>kb or a size in number of log statements?
>
>It size in bytes.

The size is in bytes.


>>  Because, if what I read was correct, in AsyncAppender, BufferSize is a 
>> number of log statements.
>
>Correct.  In AsyncAppender, BufferSize is the number of log statements.
>
>>Pascal
>
>--
>Ceki Gülcü

-- 
Ceki Gülcü

      For log4j documentation consider "The complete log4j manual"
      ISBN: 2970036908 http://www.qos.ch/shop/products/clm_t.jsp

      import org.apache.Facetime;
      ApacheCon US 2003, 18-21 November http://apachecon.com/



---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: log4j-user-help@jakarta.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: log4j-user-help@jakarta.apache.org


Mime
View raw message