lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "vivek sar" <vivex...@gmail.com>
Subject Re: DefaultIndexAccessor
Date Thu, 28 Feb 2008 23:11:39 GMT
Mark,

 Some more information,

      1) I run indexwriter every 5 mins
      2) After every cycle I check if I need to partition (based on
the index size)
      3) In the partition interface,
            a)  I first call close on the index accessor (so all the
searchers can close before I move that index)
                          accessor =
IndexAccessorFactory.getInstance().getAccessor(dir.getFile());
                          accessor.close();
            b) Then I re-open the index accessor,
                           accessor = indexFactory.getAccessor(dir.getFile());
                           accessor.open();
            c) I optimized the my indexes using the Index Writer (that
I get from the accessor).
                           masterWriter = this.indexAccessor.getWriter(false);
                           masterWriter.optimize(optimizeSegment);
            d) Once the optimization is done I release the masterWriter,
                            this.indexAccessor.release(masterWriter);

         Now here is where I get the "RejectedExecutionException".
Reading up little more on this exception,
http://pveentjer.wordpress.com/2008/02/06/are-you-dealing-with-the-rejectedexecutionexception/,
I see this might be happening because something got stuck during the
close cycle, so the ExecutorSerivce is not accepting any new tasks.
I'm not sure how would this happen.

The critical problem is once I get this exception, every release call
throws the same exception (looks like shutdown never gets done).
Because of this my readers are never refreshed and I can not read any
new indexes.

May be I've to check whether the accessor is completely closed before
re-opening?  Could you in your release check whether the pool
(ExecutorService) is in shutdown state? Any thing else I can check?

Thanks,
-vivek

On Thu, Feb 28, 2008 at 1:26 PM, vivek sar <vivextra@gmail.com> wrote:
> Mark,
>
>   We deployed our indexer (using defaultIndexAccessor) on one of the
>  production site and getting this error,
>
>  Caused by: java.util.concurrent.RejectedExecutionException
>         at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(Unknown
>  Source)
>         at java.util.concurrent.ThreadPoolExecutor.reject(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor.execute(Unknown Source)
>         at org.apache.lucene.indexaccessor.DefaultIndexAccessor.release(DefaultIndexAccessor.java:514)
>
>
>  This is happening repeatedly every time the indexer runs.
>
>  This is running your latest IndexAccessor-021508 code.  Any ideas
>  (it's kind of urgent for us)?
>
>  Thanks,
>  -vivek
>
>
>
>
>  On Fri, Feb 15, 2008 at 6:50 PM, vivek sar <vivextra@gmail.com> wrote:
>  > Mark,
>  >
>  >  Thanks for the quick fix. Actually, it is possible that there might
>  >  had been simultaneous queries using the MultiSearcher. I assumed it
>  >  was thread-safe, thus was re-using the same instance. I'll update my
>  >  application code as well.
>  >
>  >  Thanks,
>  >  -vivek
>  >
>  >
>  >
>  >  On Feb 15, 2008 5:56 PM, Mark Miller <markrmiller@gmail.com> wrote:
>  >  > Here is the fix: https://issues.apache.org/jira/browse/LUCENE-1026
>  >  >
>  >  >
>  >  > vivek sar wrote:
>  >  > > Mark,
>  >  > >
>  >  > >    There seems to be some issue with DefaultMultiIndexAccessor.java.
I
>  >  > > got following NPE exception,
>  >  > >
>  >  > >      2008-02-13 07:10:28,021 ERROR [http-7501-Processor6] ReportServiceImpl
-
>  >  > > java.lang.NullPointerException
>  >  > >         at org.apache.lucene.indexaccessor.DefaultMultiIndexAccessor.release(DefaultMultiIndexAccessor.java:89)
>  >  > >
>  >  > > Looks like the IndexAccessor for one of the Searcher in the
>  >  > > MultiSearcher returned null. Not sure how is that possible, any ideas
>  >  > > how is that possible?
>  >  > >
>  >  > > In my case it caused a critical error as the writer thread was stuck
>  >  > > forever (we found out after couple of days) because of this,
>  >  > >
>  >  > > "PS thread 9" prio=1 tid=0x00002aac70eb95d0 nid=0x6ba in Object.wait()
>  >  > > [0x0000000047533000..0x0000000047533b80]
>  >  > >         at java.lang.Object.wait(Native Method)
>  >  > >         - waiting on <0x00002aab3e5c7700> (a
>  >  > > org.apache.lucene.indexaccessor.DefaultIndexAccessor)
>  >  > >         at java.lang.Object.wait(Unknown Source)
>  >  > >         at org.apache.lucene.indexaccessor.DefaultIndexAccessor.waitForReadersAndCloseCached(DefaultIndexAccessor.java:593)
>  >  > >         at org.apache.lucene.indexaccessor.DefaultIndexAccessor.release(DefaultIndexAccessor.java:510)
>  >  > >         - locked <0x00002aab3e5c7700> (a
>  >  > > org.apache.lucene.indexaccessor.DefaultIndexAccessor)
>  >  > >
>  >  > > The only way to recover was to re-start the application.
>  >  > >
>  >  > > I use both MultiSearcher and IndexSearcher in my application, I've
>  >  > > looked at your code but not able to pinpoint how can it go wrong? Of
>  >  > > course, you do have to check for null in the
>  >  > > MultiIndexAccessor.release, but how could you get null index accessor
>  >  > > at first place?
>  >  > >
>  >  > > I do call IndexAccessor.close during partitioning of indexes, but the
>  >  > > close should wait for all Searchers to close before doing anything.
>  >  > >
>  >  > > Do you have any updates to your code since 02/04/2008?
>  >  > >
>  >  > > Thanks,
>  >  > > -vivek
>  >  > >
>  >  > > On Feb 6, 2008 8:37 AM, Jay <yu@ai.sri.com> wrote:
>  >  > >
>  >  > >> Thanks for your clarifications, Mark!
>  >  > >>
>  >  > >>
>  >  > >> Jay
>  >  > >>
>  >  > >>
>  >  > >> Mark Miller wrote:
>  >  > >>
>  >  > >>>> 5. Although currently IndexSearcher.close() does almost
nothing except
>  >  > >>>> to close the internal index reader, it might be a safer
to close
>  >  > >>>> searcher itself as well in closeCachedSearcher(), just in
case, the
>  >  > >>>> searcher may have other resources to release in the future
version of
>  >  > >>>> Lucene.
>  >  > >>>>
>  >  > >>> Didn't catch that "as well". You are right, great idea Jay,
thanks.
>  >  > >>>
>  >  > >>> ---------------------------------------------------------------------
>  >  > >>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>  >  > >>> For additional commands, e-mail: java-user-help@lucene.apache.org
>  >  > >>>
>  >  > >> ---------------------------------------------------------------------
>  >  > >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>  >  > >> For additional commands, e-mail: java-user-help@lucene.apache.org
>  >  > >>
>  >  > >>
>  >  > >>
>  >  > >
>  >  > > ---------------------------------------------------------------------
>  >  > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>  >  > > For additional commands, e-mail: java-user-help@lucene.apache.org
>  >  > >
>  >  > >
>  >  > >
>  >  >
>  >  > ---------------------------------------------------------------------
>  >  > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>  >  > For additional commands, e-mail: java-user-help@lucene.apache.org
>  >  >
>  >  >
>  >
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message