lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Burton <bur...@rojo.com>
Subject Performance tuning and org.apache.lucene.store.InputStream.BUFFER_SIZE
Date Wed, 01 Jun 2005 19:00:44 GMT

I was doing a JProfiler install of our webapp/lucene last week and of 
course a large part of our app is spent in RandomAccessFile.readBytes ...

This is called by InputStream.readByte which internally uses a 
BUFFER_SIZE of 1024 (which is the default).

This value seems too small for a default. Most filesystem block sizes 
are 4096 and a lot of controllers will refuse to read in block sizes 
smaller than 16384.

On top of that you also have the the physical disks that have a minimum 
block size and also RAID controller block sizes.

I tried to tune this variable this weekend with different settings and 
had strange results. Mostly because (I'm assuming) that my Linux 
filesystem cache and RAID controller cache were getting in the way (and 
right now I have no way to fix this).

So my questions are:

1.  What do people thing about changing this value?  Does anyone else do it?

2.  Has anyone run any benchmarks here?  Does anyone have any solid 
lucene benchmarks they can run with before/after runs with this var changed?

3.  Does anyone else think that 1024 is too small of a default value for 
modern systems?

4. Does anyone know what the default block size for other filesystems? I 
know that XFS is 4096.  What about ext2? ext3? JFS? ReiserFS? NTFS? UFS? 
etc....

Kevin

-- 


Use Rojo (RSS/Atom aggregator)! - visit http://rojo.com. 
See irc.freenode.net #rojo if you want to chat.

Rojo is Hiring! - http://www.rojonetworks.com/JobsAtRojo.html

   Kevin A. Burton, Location - San Francisco, CA
      AIM/YIM - sfburtonator,  Web - http://peerfear.org/
GPG fingerprint: 5FB2 F3E2 760E 70A8 6174 D393 E84D 8D04 99F1 4412 


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message