lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Elschot <>
Subject Re: Uneffective writeBytes and readBytes [FIX]
Date Thu, 08 Sep 2005 21:32:49 GMT
On Thursday 08 September 2005 23:17, Doug Cutting wrote:
> I don't in general disagree with this sort of optimization, but I think 
> a good fix is a bit more complicated than what you posted.
> Lukas Zapletal wrote:
> > And here comes the fixes:
> > 
> > There is significant time improvement for writing and slight for
> > reading. I also recommend set the buffer to 8 or 16 kilobytes.
> In certain cases Lucene allocates many stream buffers.  Making these 
> larger can thus greatly increase the amount of memory used.  Also, the 
> filesystem should optimize sequential reads so that the primary 
> improvement seen with a larger buffer size is fewer system calls.  In my 
> experience, a buffer of 1k or so is usually large enough so that the 
> system call overheads are minimal.
I suppose one of these cases are when many terms are used in a query. 
Would it be easily possible to make the buffer size for a term iterator
depend on the numbers of documents to be iterated?
Many terms only occur in a few documents, so this could be a 
nice win on total buffer size for the many terms case.

Paul Elschot

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message