lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Busch <>
Subject Re: [jira] Resolved: (LUCENE-709) [PATCH] Enable application-level management of IndexWriter.ramDirectory size
Date Wed, 22 Nov 2006 18:47:47 GMT
Ning Li wrote:
> I was away so I'm catching up.
> If this (occasional large documents consume too much memory) happens
> to a few applications, should it be solved in IndexWriter?
> A possible design could be:
> First, in addDocument(), compute the byte size of a ram segment after
> the ram segment is created. In the synchronized block, when the newly
> created segment is added to ramSegmentInfos, also add its byte size to
> the total byte size of ram segments.
> Then, in maybeFlushRamSegments(), either one of two conditions can
> trigger a flush: number of ram segments reaching maxBufferedDocs, and
> total byte size of ram segments exceeding a threshold.
> The overhead is very small in this design. Of course, IndexWriter
> would have another configurable parameter. :-) But it's nice if an
> application could set a limit on the memory it uses to buffer docs.
> Ning

This is exactly how I implemented it in my private version a couple of 
weeks ago. It works good and I don't see performance problems with this 
design. I named the new parameter in IndexWriter: setMaxBufferSize(long).


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message