lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Hatcher <e...@ehatchersolutions.com>
Subject Re: Segments creation
Date Mon, 24 Apr 2006 08:38:48 GMT
To be honest, I have to look up what has been written on this topic  
to get the answer right myself.  Otis did a thorough write-up in  
"Lucene in Action".  Lucene's javadocs also cover these factors here:

	<http://lucene.apache.org/java/docs/api/org/apache/lucene/index/ 
IndexWriter.html>

look at the setters, particularly setMaxMergeDocs.

	Erik


On Apr 23, 2006, at 9:47 PM, John Paige wrote:

> Thanks for the responses. For fault tolerance, we have decided to  
> set the
> "mergeFactor" to 1, ie, we want to update the on-disk index every  
> time a
> document is added via addDocument(). My question was that will this  
> create a
> new segment every time? Or is there a separate "segmentFactor",  
> such that an
> incremental index update updates an existing segment *on disk*, only
> starting a new segment once a given number of documents have been  
> added to
> the index?
>
> Thanks,
> John
>
> On 4/23/06, Erik Hatcher <erik@ehatchersolutions.com> wrote:
>>
>> If you use the compound format, all "files" are kept inside a single
>> filesystem file.
>>
>>         Erik
>>
>>
>> On Apr 23, 2006, at 2:13 PM, John Paige wrote:
>>
>>> So, if I use one indexwriter instance to index one document, will
>>> it create
>>> a segment per document?
>>> How many files per segment get added if I use compound index file
>>> format?
>>>
>>> Thanks,
>>> John
>>>
>>> On 4/23/06, Erik Hatcher <erik@ehatchersolutions.com> wrote:
>>>>
>>>> If you use the compound index file format (the default since Lucene
>>>> 1.4) you'll avoid the file descriptors issue.  If you add 10
>>>> documents at one time with a single IndexWriter, you will not  
>>>> create
>>>> 10 segments, only one segment (generally speaking, based on the
>>>> default segment factors).
>>>>
>>>>         Erik
>>>>
>>>>
>>>> On Apr 23, 2006, at 9:48 AM, John Paige wrote:
>>>>
>>>>> Hello all,
>>>>>    In my application it is required to build an index for each
>>>>> user. We need
>>>>> to add documents to the existing index frequently.
>>>>> We cannot use RAMDirectory to create a RAM index and merge it
>>>>> with the
>>>>> FSDirectory index later on based on the mergefactor. We need to  
>>>>> add
>>>>> each
>>>>> document in the FSDirectory based index.
>>>>> We are afraid of getting into the "out of file descriptors"  
>>>>> issue ,
>>>>> so my
>>>>> question is, does lucene create a segment for each document that
>>>>> gets added
>>>>> to the FS based index i.e. will adding 10 documents to the  
>>>>> existing
>>>>> index
>>>>> create 10 segments?
>>>>>
>>>>> Thanks,
>>>>> John
>>>>
>>>>
>>>> ------------------------------------------------------------------- 
>>>> --
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message