lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <markrmil...@gmail.com>
Subject Re: DefaultIndexAccessor
Date Sat, 16 Feb 2008 01:38:30 GMT
Okay, sorry about this one vivek. Added to the unit tests to expose 
this. When I took out the MultiSearcher caching, I kept the concept of 
sharing a single MultiIndexAccessor. Unfortunately, this meant that 
multiple threads were sharing the same Searcher to Accessor Map that was 
used to track which Accessor needs to release which Searcher. Because of 
this, a thread might come along and pull a Searcher out of that Map 
right before another Thread tries to release that same cached Searcher 
instance. The result is that it is not there, and hence the 
NullPointerException.

Nice to have this added to the Unit Tests. The fix is to create a new 
MultiIndexAccessor on every request, and recommend you get one for each 
Thread as the class is now not thread safe. Construction of a 
MultiIndexAccessor is pretty much nothing in terms of time. This way 
each thread has its own Map of Searcher to Accessors.

A good tip is to make a simple page that simply prints out the 
Searcher/Writer use counts. You can then check this page occasionally 
and see if it appears there are Writers or Searchers stuck out you know 
you have a problem. Under normal circumstances it should not be possible 
and indicates a bug somewhere.

I will the fix shortly.

- Mark

vivek sar wrote:
> Mark,
>
>    There seems to be some issue with DefaultMultiIndexAccessor.java. I
> got following NPE exception,
>
>      2008-02-13 07:10:28,021 ERROR [http-7501-Processor6] ReportServiceImpl -
> java.lang.NullPointerException
>         at org.apache.lucene.indexaccessor.DefaultMultiIndexAccessor.release(DefaultMultiIndexAccessor.java:89)
>
> Looks like the IndexAccessor for one of the Searcher in the
> MultiSearcher returned null. Not sure how is that possible, any ideas
> how is that possible?
>
> In my case it caused a critical error as the writer thread was stuck
> forever (we found out after couple of days) because of this,
>
> "PS thread 9" prio=1 tid=0x00002aac70eb95d0 nid=0x6ba in Object.wait()
> [0x0000000047533000..0x0000000047533b80]
>         at java.lang.Object.wait(Native Method)
>         - waiting on <0x00002aab3e5c7700> (a
> org.apache.lucene.indexaccessor.DefaultIndexAccessor)
>         at java.lang.Object.wait(Unknown Source)
>         at org.apache.lucene.indexaccessor.DefaultIndexAccessor.waitForReadersAndCloseCached(DefaultIndexAccessor.java:593)
>         at org.apache.lucene.indexaccessor.DefaultIndexAccessor.release(DefaultIndexAccessor.java:510)
>         - locked <0x00002aab3e5c7700> (a
> org.apache.lucene.indexaccessor.DefaultIndexAccessor)
>
> The only way to recover was to re-start the application.
>
> I use both MultiSearcher and IndexSearcher in my application, I've
> looked at your code but not able to pinpoint how can it go wrong? Of
> course, you do have to check for null in the
> MultiIndexAccessor.release, but how could you get null index accessor
> at first place?
>
> I do call IndexAccessor.close during partitioning of indexes, but the
> close should wait for all Searchers to close before doing anything.
>
> Do you have any updates to your code since 02/04/2008?
>
> Thanks,
> -vivek
>
> On Feb 6, 2008 8:37 AM, Jay <yu@ai.sri.com> wrote:
>   
>> Thanks for your clarifications, Mark!
>>
>>
>> Jay
>>
>>
>> Mark Miller wrote:
>>     
>>>> 5. Although currently IndexSearcher.close() does almost nothing except
>>>> to close the internal index reader, it might be a safer to close
>>>> searcher itself as well in closeCachedSearcher(), just in case, the
>>>> searcher may have other resources to release in the future version of
>>>> Lucene.
>>>>         
>>> Didn't catch that "as well". You are right, great idea Jay, thanks.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>       
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>>     
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>   

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message