lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Elschot <paul.elsc...@xs4all.nl>
Subject Re: Uneffective writeBytes and readBytes [FIX]
Date Thu, 08 Sep 2005 21:32:49 GMT
On Thursday 08 September 2005 23:17, Doug Cutting wrote:
> I don't in general disagree with this sort of optimization, but I think 
> a good fix is a bit more complicated than what you posted.
> 
> Lukas Zapletal wrote:
> > And here comes the fixes:
> > 
...
> 
> > There is significant time improvement for writing and slight for
> > reading. I also recommend set the buffer to 8 or 16 kilobytes.
> 
> In certain cases Lucene allocates many stream buffers.  Making these 
> larger can thus greatly increase the amount of memory used.  Also, the 
> filesystem should optimize sequential reads so that the primary 
> improvement seen with a larger buffer size is fewer system calls.  In my 
> experience, a buffer of 1k or so is usually large enough so that the 
> system call overheads are minimal.
> 
I suppose one of these cases are when many terms are used in a query. 
Would it be easily possible to make the buffer size for a term iterator
depend on the numbers of documents to be iterated?
Many terms only occur in a few documents, so this could be a 
nice win on total buffer size for the many terms case.

Regards,
Paul Elschot


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message