Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Message-ID: <2162093.1178279355811.JavaMail.jira@brutus>
Date: Fri, 4 May 2007 04:49:15 -0700 (PDT)
From: =?utf-8?Q?Jo=C3=A3o_Fonseca_=28JIRA=29?= <jira@apache.org>
To: java-dev@lucene.apache.org
Subject: [jira] Commented: (LUCENE-874) Automatic reopen of
 IndexSearcher/IndexReader
In-Reply-To: <6465604.1178212335620.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/LUCENE-874?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12493656 ]=20

Jo=C3=A3o Fonseca commented on LUCENE-874:
-------------------------------------

AutofreshedSearcher seems to be something like what I was proposing, but it=
 seems to rely on the NotifiableIndex mechanism. Will this work if the task=
 that updates the index is on another process/JVM?

I struggled with this problem on my own project, and came with a solution w=
ith points similar to Lucene-550 - a LuceneFactory, with factory methods fo=
r creating everything related with Lucene (IndexSearcher, IndexWriter, Anal=
yzer, etc). The factory method for creating an IndexSearcher always returns=
 the same instance, as advised by the Lucene javadocs.

The problem is when the index is modified (which, in my case, is done by an=
 external process, from time to time). The IndexSearcher must be reopened. =
There are several issues to solve:

-How to test if the index was modified? That's easy: !IndexReader.isCurrent=
 && !IndexReader.locked
-When to test if the index was modified? I test it on my LuceneFactory.getI=
ndexSearcher() method, but only from time to time - it would be costly to t=
est for every search that was made.
-The index was modified; how to close the current IndexSearcher? Other proc=
esses may be still using it, or using Hits generated by it. This is the har=
dest part to solve.
    * A reference count to the IndexSearcher must be maintained by the Luce=
neFactory, to know when all parties have finished searching.
    * To maintain a reference count, these parties must have a way to notif=
y the factory that the search is finished.
    * Also, to maintain the reference count in a thread-safe manner, some l=
ocking must be used when getting and releasing the searcher (slow!)
    * How to wait for the reference count to reach 0? On another thread? Po=
lling from time to time (on each LuceneFactory.getIndexSearcher call)?

As you can see, this is not trivial at all to do correctly - and so it shou=
ld be implemented and given out-of-the-box with Lucene. Note that the above=
 description uses Lucene as a black box, maybe it's easier to implement thi=
s inside the IndexSearcher class, by updating its internal structure when t=
he index is changed.

Another way is to maintain one IndexSearcher per thread (with a ThreadLocal=
). The reopening of the IndexSearcher would be easier, but there would be s=
everal IndexSearchers on memory...


> Automatic reopen of IndexSearcher/IndexReader
> ---------------------------------------------
>
>                 Key: LUCENE-874
>                 URL: https://issues.apache.org/jira/browse/LUCENE-874
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>            Reporter: Jo=C3=A3o Fonseca
>            Priority: Minor
>
> To improve performance, a single instance of IndexSearcher should be used=
. However, if the index is updated, it's hard to close/reopen it, because m=
ultiple threads may be accessing it at the same time.
> Lucene should include an out-of-the-box solution to this problem. Either =
a new class should be implemented to manage this behaviour (singleton Index=
Searcher, plus detection of a modified index, plus safely closing and reope=
ning the IndexSearcher) or this could be behind the scenes by the IndexSear=
cher class.

--=20
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org