lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Haxby <>
Subject Re: Performance and FS block size
Date Mon, 13 Feb 2006 08:52:31 GMT
Andrzej Bialecki wrote:

> None of you mentioned yet the aspect that 4k is the memory page size 
> on IA32 hardware. This in itself would favor any operations using 
> multiple of this size, and penalize operations using amounts below 
> this size.

For normal I/O it will rarely make any difference at all: the return 
results from read(2) are copied from kernel space to user space.   Under 
some rare conditions it can make a difference if the copy causes a page 
fault for user-space memory, but that can happen with any buffer size.   
Memory-mapped I/O does take into account the VM page size, but that's 
entirely in the kernel's domain.   I believe (though I haven't checked 
lately) that memory mapping does avoid the final copy, and it certainly 
does avoid system calls so it has the potential to be as fast as the 
undelying I/O subsystem allows it to be.   However, there are a few 
pathological cases where memory-mapped I/O is slower and you have to be 
very careful about the size of the file you're dealing with (unless 
you're running in a 64 bit process).

As Paul Elschot mentioned, the design of Lucene is the most important 
thing: it knows about locality of reference and does the right thing.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message