lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chuck Williams <ch...@manawiz.com>
Subject Re: Dynamically varying maxBufferedDocs
Date Thu, 09 Nov 2006 19:24:16 GMT
This sounds good.  Michael, I'd love to see your patch,

Chuck


Michael Busch wrote on 11/09/2006 09:13 AM:
> I had the same problem with large documents causing memory problems. I
> solved this problem by introducing a new setting in IndexWriter
> setMaxBufferSize(long). Now a merge is either triggered when
> bufferedDocs==maxBufferedDocs *or* the size of the bufferedDocs >=
> maxBufferSize. I made these changes based on the new merge policy
> Yonik mentioned, so if anyone is interested I could open a Jira issue
> and submit a patch.
>
> - Michael
>
>
> Yonik Seeley wrote:
>> On 11/9/06, Chuck Williams <chuck@manawiz.com> wrote:
>>> Thanks Yonik!  Poor wording on my part.  I won't vary maxBufferedDocs,
>>> just am making flushRamSegments() public and calling it externally
>>> (properly synchronized), earlier than it would otherwise be called from
>>> ongoing addDocument-driven merging.
>>>
>>> Sounds like this should work.
>>
>> Yep.
>> For best behavior, you probably want to be using the current
>> (svn-trunk) version of Lucene with the new merge policy.  It ensures
>> there are mergeFactor segments with size <= maxBufferedDocs before
>> triggering a merge.  This makes for faster indexing in the presence of
>> deleted docs or partially full segments.
>>
>> -Yonik
>> http://incubator.apache.org/solr Solr, the open-source Lucene search
>> server
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message