Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: domain of DORONC@il.ibm.com designates
 195.212.29.150 as permitted sender)
In-Reply-To: <5225348.post@talk.nabble.com>
Subject: Re: modify existing non-indexed field
To: java-user@lucene.apache.org
Message-ID: 
 <OF71C3F89B.4CC9A6C3-ON882571A4.007A341D-882571A4.007DA4EF@il.ibm.com>
From: Doron Cohen <DORONC@il.ibm.com>
Date: Fri, 7 Jul 2006 15:52:22 -0700
MIME-Version: 1.0
Content-type: text/plain; charset=US-ASCII

> dan2000 <liudan@ntlworld.com> wrote on 07/07/2006 15:03:35:
> but if you remove it and add it again, you'll need to re-index it again.
> don't you? When you do re-index, you'll have to close the reader, which
> would pause the search. Any better way of doint it?

INHO yes and no -

There's no need to close the searcher (or the reader associated with the
searcher) in order to add that doc again, however it is required to close
the reader that is used for deleting a document in order to open a writer
for adding the updated document. Also, in order for this document to become
visible to search, a new searcher must be opened. This does not necessarily
means "pausing the search" - it is possible to open a new searcher in the
background (as done in Solr) and when it is ready - to switch to serve new
queries with the new searcher and close (and free) the old searcher once
its serviced queries are "completed". Solr (
http://incubator.apache.org/solr) does something similar, and it also warms
newly created searchers.

If the the update of these non-idexed field happens a lot, perhaps a side
data store would be more adequate, i.e. Lucene would store a unique docID
(non-indexed field) for each document, and at search time that ID would be
used to look-up the (updatable) values in the side data store. This means
additional code complexity and, search performance penalty if many result
documents are of interest for every search.

The approach of updating the documents <delete/add> moves the overhead to
indexing while making search faster and the application code simpler. If
this is the selected approach, especially if the updates cannot be easily
"batched" (i.e. delete 1000 docs to be updated, add back those updated 1000
docs), then the following may be of interest - a related proposal to "allow
updating documents with no need to repeatedly open/close reader/writer":
- http://issues.apache.org/jira/browse/LUCENE-565
- http://www.gossamer-threads.com/lists/lucene/java-dev/35317

Hope this helps,
Doron


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org