lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: IndexWriter.close() no longer seems to close everything
Date Fri, 13 Nov 2009 10:33:53 GMT
Any luck narrowing this to a standalone test case that shows the problem?

That new exception appears to be inside the Java code created by the
app server compiling your JSP -- it's not very helpful since it
doesn't "enter" Lucene.  Can you try to narrow this to a standalone
test case, too?

Thanks.

Mike

On Fri, Nov 13, 2009 at 5:12 AM, Albert Juhe <albertjuhe@gmail.com> wrote:
>
> Hi,
>
> About this problem I did a test yesterday, I did a downgrade, I changed
> versión 2.9.1 to 2.4.1, and the problem has been solved, all the files are
> closed corretly and JBOSS isn't unstable.
>
> Another problem that we have observed is:
>
> Sometimes, random success, when you try to make a serach the first time
> don't return result but the second time yes (Lucene 2.9.1. BooleanQuery).
> I get this error message:
> java.lang.NullPointerException
>        at
> org.apache.jsp.objects.buscar_005fresposta_jsp._jspService(buscar_005fresposta_jsp.java:631)
>        at
> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
>        at
> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:334)
>        at
> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:314)
>        at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:264)
>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
>        at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
>        at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
>        at
> org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
>        at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
>        at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
>        at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
>        at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
>        at
> org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:175)
>        at
> org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:74)
>        at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
>        at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
>        at
> org.jboss.web.tomcat.tc5.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:156)
>        at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
>        at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
>        at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
>        at
> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
>        at
> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
>        at
> org.apache.tomcat.util.net.MasterSlaveWorkerThread.run(MasterSlaveWorkerThread.java:112)
>        at java.lang.Thread.run(Thread.java:595)
>
> I don't know whats the meanig of it, but buscar_resposta.jsp is the jsp that
> makes a Search.
>
> With 2.4.1 this problem don't happens.
>
>
> Jason Rutherglen-2 wrote:
>>
>> If there's a bug you're seeing, it's helpful to open an issue and post
>> code reproducing it.
>>
>> On Wed, Nov 11, 2009 at 3:41 AM, Albert Juhe <albertjuhe@gmail.com> wrote:
>>>
>>> I think that this is the best way to proceed.
>>>
>>> thank you Mike
>>>
>>>
>>>
>>> Michael McCandless-2 wrote:
>>>>
>>>> Can you narrow the leak down to a small self-contained test?
>>>>
>>>> Mike
>>>>
>>>> On Wed, Nov 11, 2009 at 5:54 AM, Albert Juhe <albertjuhe@gmail.com>
>>>> wrote:
>>>>>
>>>>> I don't get any exception.
>>>>>
>>>>> thank you Mike
>>>>>
>>>>>
>>>>> Michael McCandless-2 wrote:
>>>>>>
>>>>>> Do you see your exception handler printing anything out?
>>>>>>
>>>>>> You don't need to close the underlying IndexReader, just the
>>>>>> IndexSearcher (which will close the IndexReader, since it was the
one
>>>>>> that had opened it).
>>>>>>
>>>>>> Mike
>>>>>>
>>>>>> On Wed, Nov 11, 2009 at 5:10 AM, Albert Juhe <albertjuhe@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> I don't know if it's the same problem but I think it's similar,
>>>>>>>
>>>>>>> My problem is with the Indexsearcher. I've installed a web search
>>>>>>> engine
>>>>>>> that uses Lucene, after a search I make a close operation like
this:
>>>>>>>
>>>>>>> private IndexSearcher searcher;
>>>>>>>
>>>>>>> NIOFSDirectory directory = new NIOFSDirectory(new File(path));
>>>>>>> this.searcher = new IndexSearcher(directory, true);
>>>>>>>
>>>>>>>  public void close() {
>>>>>>>        try {
>>>>>>>            System.out.println("Closing: "
>>>>>>> +this.searcher.getIndexReader().directory().toString() );
>>>>>>>            this.searcher.getIndexReader().directory().close();
>>>>>>>            this.searcher.getIndexReader().close();
>>>>>>>            this.searcher.close();
>>>>>>>        } catch (IOException e) {
>>>>>>>            System.out.println(" caught a " + e.getClass()
+ "\n with
>>>>>>> message: " + e.getMessage());
>>>>>>>        }
>>>>>>>    }
>>>>>>> jboss don't close *.cfs index files, everytime that I make a
new
>>>>>>> search
>>>>>>> a
>>>>>>> new file is caught by jboss.
>>>>>>> If I make 5 searchs jboss caught 5 *.cfs, I see it with lsof
command,
>>>>>>> the
>>>>>>> only way to free this files and close it is stopping jboss.
>>>>>>>
>>>>>>> The problem is that after search many times,  jboss go down
and I
>>>>>>> have
>>>>>>> to
>>>>>>> restart it.
>>>>>>>
>>>>>>> Do you have the same problem with IndexSearch or is only with
>>>>>>> IndexWriter?
>>>>>>>
>>>>>>> Albert Juhe
>>>>>>> Learning Technologies
>>>>>>> Universitat Oberta de Catalunya
>>>>>>>
>>>>>>>
>>>>>>> Michael McCandless-2 wrote:
>>>>>>>>
>>>>>>>> Does this look like a real leak John?  You're definitely
closing
>>>>>>>> every
>>>>>>>> reader you get back from getReader?
>>>>>>>>
>>>>>>>> Mike
>>>>>>>>
>>>>>>>> On Sun, Nov 8, 2009 at 10:41 PM, John Wang <john.wang@gmail.com>
>>>>>>>> wrote:
>>>>>>>>> I am seeing the samething, but only when IndexWriter.getReader
is
>>>>>>>>> called
>>>>>>>>> at
>>>>>>>>> a high rate.
>>>>>>>>>
>>>>>>>>> from lsof, I see file handles growing.
>>>>>>>>>
>>>>>>>>> -John
>>>>>>>>>
>>>>>>>>> On Sun, Nov 8, 2009 at 7:29 PM, Daniel Noll <daniel@nuix.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi all.
>>>>>>>>>>
>>>>>>>>>> We updated to Lucene 2.9, and now we find that after
closing our
>>>>>>>>>> text
>>>>>>>>>> index, it is not possible to rename the directory
in which it
>>>>>>>>>> resides
>>>>>>>>>> (we are actually renaming a directory further up
the hierarchy.)
>>>>>>>>>>
>>>>>>>>>> We discovered that the following files were still
open by the
>>>>>>>>>> process:
>>>>>>>>>>
>>>>>>>>>>  _0.tis, _0.frq, _0.prx, _0.fdt, _0.fdx, _0.tvx,
_0.tvd, _0.tvf
>>>>>>>>>>
>>>>>>>>>> We are calling IndexWriter.close() shortly before
attempting to
>>>>>>>>>> write
>>>>>>>>>> to the directory (a few lines of code earlier) so
I suspect it
>>>>>>>>>> could
>>>>>>>>>> be related to timing somehow if Lucene is perhaps
still doing
>>>>>>>>>> something on a background thread at this time (though
I was under
>>>>>>>>>> the
>>>>>>>>>> impression that close() waited for merges and so
forth to complete
>>>>>>>>>> before returning.)
>>>>>>>>>>
>>>>>>>>>> Daniel
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Daniel Noll                          
 Forensic and eDiscovery
>>>>>>>>>> Software
>>>>>>>>>> Senior Developer                      
       The world's most
>>>>>>>>>> advanced
>>>>>>>>>> Nuix                              
                 email data
>>>>>>>>>> analysis
>>>>>>>>>> http://nuix.com/                      
         and eDiscovery
>>>>>>>>>> software
>>>>>>>>>>
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> View this message in context:
>>>>>>> http://old.nabble.com/IndexWriter.close%28%29-no-longer-seems-to-close-everything-tp26260801p26298910.html
>>>>>>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> http://old.nabble.com/IndexWriter.close%28%29-no-longer-seems-to-close-everything-tp26260801p26299405.html
>>>>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/IndexWriter.close%28%29-no-longer-seems-to-close-everything-tp26260801p26299941.html
>>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>>
>
> --
> View this message in context: http://old.nabble.com/IndexWriter.close%28%29-no-longer-seems-to-close-everything-tp26260801p26334253.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message