lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <>
Subject Re: detected corrupted index / performance improvement
Date Fri, 08 Feb 2008 18:49:22 GMT
robert engels wrote:
> But that would mean we should be using at least 250k buffers for the 
> IndexInput ? Not the 16k or so that is the default.
> Is the OS smart enough to figure out that the file is being sequentially 
> read, and adjust its physical read size to 256k, based on the other 
> concurrent IO operations. Seems this would be hard for it to figure out, 
> and have it not perform poorly in the general case.

Benchmarks have shown that OSes do a decent job at this.  You can 
increase the applications buffer sizes, but you might just end up 
wasting memory if the OS is already doing the right thing.  The linux 
kernel dynamically increases the readahead window based on the access 
pattern: the more you read sequentially, the larger the readahead window.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message