lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Busch <>
Subject Re: Dynamically varying maxBufferedDocs
Date Thu, 09 Nov 2006 19:13:09 GMT
I had the same problem with large documents causing memory problems. I 
solved this problem by introducing a new setting in IndexWriter 
setMaxBufferSize(long). Now a merge is either triggered when 
bufferedDocs==maxBufferedDocs *or* the size of the bufferedDocs >= 
maxBufferSize. I made these changes based on the new merge policy Yonik 
mentioned, so if anyone is interested I could open a Jira issue and 
submit a patch.

- Michael

Yonik Seeley wrote:
> On 11/9/06, Chuck Williams <> wrote:
>> Thanks Yonik!  Poor wording on my part.  I won't vary maxBufferedDocs,
>> just am making flushRamSegments() public and calling it externally
>> (properly synchronized), earlier than it would otherwise be called from
>> ongoing addDocument-driven merging.
>> Sounds like this should work.
> Yep.
> For best behavior, you probably want to be using the current
> (svn-trunk) version of Lucene with the new merge policy.  It ensures
> there are mergeFactor segments with size <= maxBufferedDocs before
> triggering a merge.  This makes for faster indexing in the presence of
> deleted docs or partially full segments.
> -Yonik
> Solr, the open-source Lucene search 
> server
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message