lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From João Fonseca (JIRA) <j...@apache.org>
Subject [jira] Commented: (LUCENE-874) Automatic reopen of IndexSearcher/IndexReader
Date Fri, 04 May 2007 11:49:15 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12493656
] 

João Fonseca commented on LUCENE-874:
-------------------------------------

AutofreshedSearcher seems to be something like what I was proposing, but it seems to rely
on the NotifiableIndex mechanism. Will this work if the task that updates the index is on
another process/JVM?

I struggled with this problem on my own project, and came with a solution with points similar
to Lucene-550 - a LuceneFactory, with factory methods for creating everything related with
Lucene (IndexSearcher, IndexWriter, Analyzer, etc). The factory method for creating an IndexSearcher
always returns the same instance, as advised by the Lucene javadocs.

The problem is when the index is modified (which, in my case, is done by an external process,
from time to time). The IndexSearcher must be reopened. There are several issues to solve:

-How to test if the index was modified? That's easy: !IndexReader.isCurrent && !IndexReader.locked
-When to test if the index was modified? I test it on my LuceneFactory.getIndexSearcher()
method, but only from time to time - it would be costly to test for every search that was
made.
-The index was modified; how to close the current IndexSearcher? Other processes may be still
using it, or using Hits generated by it. This is the hardest part to solve.
    * A reference count to the IndexSearcher must be maintained by the LuceneFactory, to know
when all parties have finished searching.
    * To maintain a reference count, these parties must have a way to notify the factory that
the search is finished.
    * Also, to maintain the reference count in a thread-safe manner, some locking must be
used when getting and releasing the searcher (slow!)
    * How to wait for the reference count to reach 0? On another thread? Polling from time
to time (on each LuceneFactory.getIndexSearcher call)?

As you can see, this is not trivial at all to do correctly - and so it should be implemented
and given out-of-the-box with Lucene. Note that the above description uses Lucene as a black
box, maybe it's easier to implement this inside the IndexSearcher class, by updating its internal
structure when the index is changed.

Another way is to maintain one IndexSearcher per thread (with a ThreadLocal). The reopening
of the IndexSearcher would be easier, but there would be several IndexSearchers on memory...






> Automatic reopen of IndexSearcher/IndexReader
> ---------------------------------------------
>
>                 Key: LUCENE-874
>                 URL: https://issues.apache.org/jira/browse/LUCENE-874
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>            Reporter: João Fonseca
>            Priority: Minor
>
> To improve performance, a single instance of IndexSearcher should be used. However, if
the index is updated, it's hard to close/reopen it, because multiple threads may be accessing
it at the same time.
> Lucene should include an out-of-the-box solution to this problem. Either a new class
should be implemented to manage this behaviour (singleton IndexSearcher, plus detection of
a modified index, plus safely closing and reopening the IndexSearcher) or this could be behind
the scenes by the IndexSearcher class.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message