Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: domain of NYH@il.ibm.com designates
 195.212.29.151 as permitted sender)
In-Reply-To: <c68e39170602210713u353d5917h412cd3e416736ffc@mail.gmail.com>
Subject: Re: Open an IndexWriter in parallel with an IndexReader on the same
 index.
To: java-user@lucene.apache.org
Message-ID: 
 <OFBC0D871B.E7CC2A41-ONC225711C.0054C492-C225711C.0055B04E@il.ibm.com>
From: "Nadav Har'El" <NYH@il.ibm.com>
Date: Tue, 21 Feb 2006 17:35:56 +0200
MIME-Version: 1.0
Content-type: text/plain; charset=US-ASCII

"Yonik Seeley" <yseeley@gmail.com> wrote on 21/02/2006 05:13:52 PM:
> On 2/21/06, Pierre Luc Dupont <PLDupont@mediagrif.com> wrote:
> >     is it possible to open an IndexWriter and an IndexReader on the
same
> > index, at the same time,
> > to do deleteTerm and addDocument?
>
> No, it's not possible.  You should batch things: do all your
> deletions, close the IndexReader, then open an IndexWriter and do all
> the addDocument calls.

For some applications, the seperation of indexWriter (which can add a
document) and indexReader (which can delete a document) is very
inconvenient.
For example, consider a case where documents are often updated, and we
often need to find and remove the old document and add the new version
of the document. the "indexModifier" class nicely hides the complexity
from us and allows both addition and deletion, but the documentation
says its performance sucks (when used in the way I just outlined):
imagine 1000 documents being modified, and now we start deleting and
adding each one, one after another.

It would have been nice if someone wrote something like indexModifier,
but with a cache, similar to what Yonik suggested above: deletions will
not be done immediately, but rather cached and later done in batches.
Of course, batched deletions should not remember the term to delete,
but rather the matching document numbers at the time of the deletion -
because after the addition of the modified document if we search for
the term again we'll find two documents.

What about this idea? Does an implementation of something similar
already exist?

--
Nadav Har'El


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org