lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: Setting the maximum number of documents in a lucene segment
Date Fri, 25 May 2007 15:43:07 GMT
Hello Ard,

What you are after is a higher mergeFactor and probably also a higher maxBufferedDocs.  Is
indexing performance the concern?
Don't go crazy with setting a super high (e.g. 100+) mergeFactor, unless you really have the
number of open files on your server(s) set to a solid/high number. maxBufferedDocs can be
set to a much higher number, typically, depending on the size of the documents you are trying
to index and the amount of heap the JVM has to work with.  There is also a new API for explicit
flushes of in-memory documents while indexing to control memory consumption.

Otis
--
Lucene Consulting -- http://lucene-consulting.com/


----- Original Message ----
From: Ard Schrijvers <a.schrijvers@hippo.nl>
To: java-user@lucene.apache.org
Sent: Friday, May 25, 2007 8:40:26 AM
Subject: RE: Setting the maximum number of documents in a lucene segment


> 
> Hello,
> 
> I am trying to change the maximum number of documents in a 
> lucene segment. By default it seems to be 10.

Correction: 10 for the smallest (just created) segments of course, because obviously merged
segments are likely to contain many more documents

> When I have a 
> mergeFactor of say 10, then on average, after every 100 added 
> documents lucene is merging segments.
> 
> I want each segment to contain more then the default 10 
> documents, because I need to minimize merging.
> 
> Is there a way to achieve this? 
> writer.setMaxBufferedDocs(largeValue) does not do the trick 
> (I think because in my case because the writer is flushed and 
> closed after an few updates)
> 
> Does anyone know wether it is possible to make the default 
> number of documents a segment can contain larger?
> 
> Thanks in advance, 
> 
> Ard Schrijvers
> 
> 
> -- 
> 
> Hippo
> Oosteinde 11
> 1017WT Amsterdam
> The Netherlands
> Tel  +31 (0)20 5224466
> -------------------------------------------------------------
> a.schrijvers@hippo.nl / http://www.hippo.nl
> -------------------------------------------------------------- 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org





---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message