lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Kimm <ch...@seeqa.com>
Subject Re: update performance
Date Thu, 11 Mar 2004 18:29:38 GMT
Unfortunately, I'm not able to batch the updates.  The application needs 
to make some descisions based on what each document looks like before 
and  after the update, so I have to do it one at a time.  I guess this 
is not a common useage scenario for Lucene.  Otherwise, an update() 
might already be built in somewhere.

Is there anything in the locking/sync framework which precludes saving 
the cost of closing the Directory object and deleting the temp lock file 
each time an update is made?

-Chris

Doug Cutting wrote:

> It sounds like you're not batching your updates.
> 
> The most efficient approch to update 1000 documents would be to:
> 
>   1. Open an IndexReader;
>   2. Delete all 1000 documents.
>   3. Close the reader;
>   4. Open an IndexWriter;
>   5. Add all 1000 updated documents;
>   6. Close the IndexWriter.
> 
> Is that what you're doing?
> 
> Doug
> 
> Chris Kimm wrote:
> 
>> The standard pattern for updating an index - removing a document then 
>> re-adding the modified document to the index - is currently a 
>> significant performance bottleneck in my application.  I sometimes 
>> need to update ~1000 documents at a time.  The major cost of this 
>> pattern as far as I can see is IndexWriter.close ().   Average times 
>> for an update to an FSDirectory look like this:
>>
>> delete document: 7 ms
>> create document: 6 ms
>> add document: 11 ms
>> IndexWriter.close: 59 ms
>>
>> Is there a way to synchronize IndexWriter and IndexReader so that a 
>> call to IndexWriter.close is not required for each update?  I guess I 
>> mean to ask if there is a *simple* way to do this.  I imagine that one 
>> could write an IndexUpdater class which manages the synchronization of 
>> Locks, temp files, etc.
>>
>> Thanks,
>>
>> Chris
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message