lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Different Index Reader creation method affecting result
Date Thu, 21 Jul 2011 13:14:25 GMT
One small correction here: NRT reopen is much cheaper than IW.commit /
IR.reopen, for getting indexed changes visible to your reader, since
it doesn't invoke fsync and doesn't have to write deleted docs to the
directory only to then load them again in the reader.

But, it's still fairly costly, so you shouldn't, say, reopen your NRT
reader for every query, if you have a high query rate; it's better to
rate limit the reopens.

Mike McCandless

http://blog.mikemccandless.com

On Wed, Jul 20, 2011 at 6:02 PM, Simon Willnauer
<simon.willnauer@googlemail.com> wrote:
> On Wed, Jul 20, 2011 at 11:50 PM, Saurabh Gokhale
> <saurabhgokhale@gmail.com> wrote:
>> Hi All,
>>
>> I am using Lucene 3.1 in the project.
>>
>> *Background for the question:*
>> I am working on the application which starts with 2 threads, one performs
>> indexing activity and other performs searching activity (I create searcher
>> object from reader object). Both these threads run periodically and index or
>> search new documents if they are available.
>> My searcher searches a string in the lucene index using a BooleanQuery with
>> SHOULD clause.
>>
>>
>> *Problem:*
>> My problem is as follows:
>>
>> As I see there are 2 ways to create indexReader (to create searcher)
>>      A. indexWriter.getReader() --> to get near real time searching
>>      or
>>      B. Using directly the index directory path like indexReader =
>> IndexReader.open(NIOFSDirectory.open(new File(DIR_PATH)), true)
>>
>> Now when the application starts, since there is no index created yet if I
>> use option "B" to create indexReader, I fail with IndexNotFoundException.
>> If I create IndexReader using option "A", I dont fail with exception but do
>> not get any result for my input which is ok since I dont have searchable
>> index yet.
>>
>> My problem is, with option "A" of creating reader object, even after
>> indexing say 1000 documents and I still dont match any result. (The index
>> does have matching results. I can say that for sure because if I stop my
>> application, change the code to create reader using option B and then search
>> the same string on the newly created index of 1000 documents, I get lot of
>> matches.).
>>
>> I am missing something due to which I am not getting any match for option A?
>> What am I doing wrong?
>
> hi there,
>
> the answer is yes, you are missing something :). When you open the
> reader from your writer via IW#getReader() you get a "current view" of
> your index (same is true when opening from directory). To refresh this
> view you need to call IndexReader#reopen() in order to seen any
> updates. For the NRT reader (created via IW#getReader()) this will be
> an uncommitted view giving you access to documetns not yet committed.
> If you are opened from a Directory you see all the already committed
> changes.
>
> you should use the reopen like this:
> IndexReader reader = ...
> IndexReader newReader = r.reopen();
> if (newReader != reader) {
>   // reader was reopened
>   reader.close();
> }
> reader = newReader;
>
> please be careful with reopening the reader, this operation can be
> costly if you are useing NRT readers. You should maybe do this
> periodically or if you don't have any freshness constraints simply use
> a reader opened from a directoy and reopen after commit.
>
> simon
>>
>>
>> thanks
>>
>> Saurabh
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message