lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Busch <busch...@gmail.com>
Subject Re: Possible IndexInput optimization
Date Sun, 29 Mar 2009 09:58:54 GMT
On 3/29/09 12:43 AM, Earwin Burrfoot wrote:
> There are three cases when we can override readNNN methods and provide
> implementations with zero or minimum method invocations -
> RAMDirectory, MMapDirectory and BufferedIndexInput for
> FSDirectory/CompoundFileReader. Anybody tried this?
>
>    

A while ago I tried overriding the read* methods in BufferedIndexInput 
like this:

     public int readVInt() throws IOException {
         if (5 <= (bufferLength-bufferPosition)) {
           return readVIntFast();
         }
         return super.readVInt();
     }

     private int readVIntFast() throws IOException {
         byte b = buffer[bufferPosition++];
         int i = b & 0x7F;
         for (int shift = 6; (b & 0x80) != 0; shift += 7) {
           b = buffer[bufferPosition++];
           i |= (b & 0x7F) << shift;
         }
         return i;
     }


Notice that I don't rely on ArrayIndexOutOfBoundsException, instead I do 
one range check in readVInt() and then call the readVIntFast() method, 
which accesses the buffer array directly to avoid multiple range checks.

Surprisingly I did not see any performance improvement. In my test I 
wrote a huge file (several GBs) to disk with VInts, making sure they 
occupied more than just a single byte each. Reading the file with and 
without this "optimization" in BufferedIndexInput made almost no 
difference. Only when I ran it in a profiler I saw a big difference, 
because with his change there are less method calls, hence less 
invocation count overhead.

I'm still surprised there was no performance improvement at all. Maybe 
something was wrong with my test and I should try it again...

-Michael

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message