lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nadav Har'El" <...@il.ibm.com>
Subject Re: Open an IndexWriter in parallel with an IndexReader on the same index.
Date Wed, 22 Feb 2006 08:23:10 GMT
Chris Hostetter <hossman_lucene@fucit.org> wrote on 22/02/2006 03:24:58 AM:
>
> : It would have been nice if someone wrote something like indexModifier,
> : but with a cache, similar to what Yonik suggested above: deletions will
> : not be done immediately, but rather cached and later done in batches.
> : Of course, batched deletions should not remember the term to delete,
> : but rather the matching document numbers at the time of the deletion -
> : because after the addition of the modified document if we search for
> : the term again we'll find two documents.
>
> That's not a safe sequence of events.  An Add can trigger a segment
merge,
> which cna renumber documents.

I see. Then maybe there's a way to catch this merge and do the deletions
just before it, because...

> As yonik said, you want to queue up the adds/updates, then do a delete
for
> each update in your queue, then do your adds in one batch.  knowing

The problem in this solution is that unlike queuing deletes, queuing
additions requires you to queue the actual document contents. Doing
this in memory might add a large memory pentalty which is undesired
for applications that try to maintain a small memory footprint.

> when/what to delete requies knowing a "key" for your records -- which
> isnt' a native lucne concept, but it is certainly a general enough one
> that a helper class could be written for this.

I realise that the name of this delete key isn't defined by Lucene,
but I believe that the concept of such a key was "officially"
sanctioned by Lucene with the deleteDocuments(Term) method (whose
documentation even mentions the "unique ID string" scenario).
So indeed a helper class of this sort will probably be useful to
more than a few people.

--
Nadav Har'El


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message