lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Veentjer - Anchor Men" <p.veent...@anchormen.nl>
Subject RE: Update performance/indexwriter.delete()?
Date Thu, 14 Apr 2005 14:50:11 GMT
 

-----Oorspronkelijk bericht-----
Van: Roy Klein [mailto:klein@sitescape.com] 
Verzonden: donderdag 14 april 2005 15:40
Aan: java-user@lucene.apache.org
Onderwerp: Update performance/indexwriter.delete()?


>>I've got an application that will be doing 
>>constant updates to an index.
>>I've looked into batching those updates, however,
>>based on the way the application works, 
>>the updates can't be batched. 

You could always batch a single document :)

At the moment I have a components that stores documents to write/delete
and is called once and a while. This makes it very easy to do batch
processing (and also makes it easier to deal with the concurrency
problems)

>>(Well, I figure with a lot of work, I might be able 
>>to batch ~10% of the transactions) Another requirement 
>>of my application is that the index needs to be updated
>> synchronously (i.e. a given thread will update a document
>>, and then immediately issue a query in which the updated 
>>document must be in the resulting hits), so keeping a 
>>"background working copy" of the index and switching 
>>it out with a live copy doesn't work in this case.

This is not possible onless you only process one single 
document at a time. After the document is writing you have to 
refresh the readed and find the document. Why do have such a
requirement? 
Are you afraid that a single document is indexed more than once? :)

In the platform for Lucene I have writen for all document it is very
very
Important that the index never contains two (or more)instances of the 
Same document. This is possible, even with doing batch updates :)

>>I started looking at the performance issues involved in updating one 
>>document at a time, and found that most of the cost is in opening 
>>and closing readers&writers.

That is why batchupdating is invented :)

>>So one thing I've been wondering:  Why do you need to do 
>>deletes from an indexreader?  Is there some reason why a writer 
>>couldn't be modified to do both deletes and adds?

>>Could someone who is familiar with the design please explain why a
reader is
>>required?   This seems to be a common enough issue that I bet a lot of
>>people on this list would benefit from a thorough explanation.   ( I
just
>>ordered the LIA book, if it's explained in there, then my apologies
for not having read it before posting here)

It is explained in that book ;)

Roy


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org





---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message