lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lukas Zapletal <lukas.zaple...@qcm.cz>
Subject Re: Uneffective writeBytes [FIXED PATCH]
Date Sun, 09 Oct 2005 22:21:05 GMT
> I don't in general disagree with this sort of optimization, but I think 
> a good fix is a bit more complicated than what you posted.

Hi Doug, I had some time to redo my fixes...

> This forces a flush() each time a byte array of any size is written. 
> That could be much slower when lots of small byte arrays are written, 
> since flush() invokes a system call.  What would be best is, if there is 

Yes, I made a better version in an exact way you have described above.
The patch replaces writeBytes method with the whole new implementation:

  public void writeBytes(byte[] b, int length) throws IOException {
    int bytesLeft = BUFFER_SIZE - bufferPosition;
    // is there enough space in the buffer?
    if (bytesLeft >= length) {
      // we add the data to the end of the buffer
      System.arraycopy(b, 0, buffer, bufferPosition, length);
      bufferPosition += length;
      // if the buffer is full, flush it
      if (BUFFER_SIZE - bufferPosition == 0)
        flush();
    } else {
      // we fill/flush the buffer (until the input is written)
      int pos = 0; // position in the input data
      int pieceLength;
      while (pos < length) {
        pieceLength = (length - pos < bytesLeft) ? 
		length - pos : bytesLeft; 
	System.arraycopy(b, pos, buffer, 
		bufferPosition, pieceLength); 
	pos += pieceLength;
        bufferPosition += pieceLength;
        // if the buffer is full, flush it
        bytesLeft = BUFFER_SIZE - bufferPosition;
        if (bytesLeft == 0) {
          flush();
          bytesLeft = BUFFER_SIZE;
        }
      }
    }
  }

This is much smarter and it is not slow for lots of one-byte writes. I
agree with the assession of my "fix" of readBytes:

> Again, this could be much slower when lots of small arrays are written, 
> since each call forces seek and read system calls.  However this could 
> be optimized for the case where the desired data resides entirely in the 
> current buffer to use System.arraycopy.

So I tried to optimize the reads from buffer, but the method was so
complicated I dropped this thought. Readbytes is fast enough.

I have modified the StoreTest, now you can test the *byte or *bytes
methods (3rd parameter). Because of creating and testing the buffers,
there is not speed improvement here but when you write from some
prepared buffer (e.g. from a file), the speed increases.

-- 
S pozdravem / Best regards
Lukas Zapletal
sefredaktor / editor-in-chief
LinuxEXPRES - opravdovy linuxovy magazin
www.LinuxEXPRES.cz
tel.:+420 777 003 843

Mime
View raw message