lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Miles Barr <mi...@runtime-collective.com>
Subject Re: Large Index managing
Date Wed, 02 Mar 2005 09:51:23 GMT
On Wed, 2005-03-02 at 05:49, Otis Gospodnetic wrote:
> Or you can just buffer your update requests and delete in batch and
> then add in batch.
> 
> Or you could keep that IndexReader on the large index and use it to
> delete objects, while doing adds on a RAMDirectory.  Then, when you are
> done, merge the RAMDirectory index with the large one.  Also just an
> untested idea.... and really something that is not much different from
> simple buffering in your application logic.

You can batch updates to an index but you cannot do all the deletes and
all the writes because there's no primary key concept in Lucene.
Updating a document consists of two actions, first removing it, then
adding the new copy. If you update a document more than once between
doing the batch changes you can end up with different versions on the
same document in the index.

e.g.

updateDoc(key, doc);

... changes to doc ...

updateDoc(key, doc);

would become

reader.delete(doc);
// 1 document deleted

reader.delete(doc);
// 0 documents deleted

writer.add(doc);
writer.add(doc);

// N.B. syntax is incorrect

and doc would appear twice

You have to do the deletes and writes in the order they happen. But at
least by batching them you can make the long wait infrequent.



-- 
Miles Barr <miles@runtime-collective.com>
Runtime Collective


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message