lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Albert Juhe <albertj...@gmail.com>
Subject Re: IndexWriter.close() no longer seems to close everything
Date Fri, 13 Nov 2009 10:12:59 GMT

Hi,

About this problem I did a test yesterday, I did a downgrade, I changed
versión 2.9.1 to 2.4.1, and the problem has been solved, all the files are
closed corretly and JBOSS isn't unstable.

Another problem that we have observed is:

Sometimes, random success, when you try to make a serach the first time
don't return result but the second time yes (Lucene 2.9.1. BooleanQuery).
I get this error message:
java.lang.NullPointerException
        at
org.apache.jsp.objects.buscar_005fresposta_jsp._jspService(buscar_005fresposta_jsp.java:631)
        at
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
        at
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:334)
        at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:314)
        at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:264)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
        at
org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
        at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
        at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
        at
org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:175)
        at
org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:74)
        at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
        at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
        at
org.jboss.web.tomcat.tc5.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:156)
        at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
        at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
        at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
        at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
        at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
        at
org.apache.tomcat.util.net.MasterSlaveWorkerThread.run(MasterSlaveWorkerThread.java:112)
        at java.lang.Thread.run(Thread.java:595)

I don't know whats the meanig of it, but buscar_resposta.jsp is the jsp that
makes a Search.

With 2.4.1 this problem don't happens.


Jason Rutherglen-2 wrote:
> 
> If there's a bug you're seeing, it's helpful to open an issue and post
> code reproducing it.
> 
> On Wed, Nov 11, 2009 at 3:41 AM, Albert Juhe <albertjuhe@gmail.com> wrote:
>>
>> I think that this is the best way to proceed.
>>
>> thank you Mike
>>
>>
>>
>> Michael McCandless-2 wrote:
>>>
>>> Can you narrow the leak down to a small self-contained test?
>>>
>>> Mike
>>>
>>> On Wed, Nov 11, 2009 at 5:54 AM, Albert Juhe <albertjuhe@gmail.com>
>>> wrote:
>>>>
>>>> I don't get any exception.
>>>>
>>>> thank you Mike
>>>>
>>>>
>>>> Michael McCandless-2 wrote:
>>>>>
>>>>> Do you see your exception handler printing anything out?
>>>>>
>>>>> You don't need to close the underlying IndexReader, just the
>>>>> IndexSearcher (which will close the IndexReader, since it was the one
>>>>> that had opened it).
>>>>>
>>>>> Mike
>>>>>
>>>>> On Wed, Nov 11, 2009 at 5:10 AM, Albert Juhe <albertjuhe@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> I don't know if it's the same problem but I think it's similar,
>>>>>>
>>>>>> My problem is with the Indexsearcher. I've installed a web search
>>>>>> engine
>>>>>> that uses Lucene, after a search I make a close operation like this:
>>>>>>
>>>>>> private IndexSearcher searcher;
>>>>>>
>>>>>> NIOFSDirectory directory = new NIOFSDirectory(new File(path));
>>>>>> this.searcher = new IndexSearcher(directory, true);
>>>>>>
>>>>>>  public void close() {
>>>>>>        try {
>>>>>>            System.out.println("Closing: "
>>>>>> +this.searcher.getIndexReader().directory().toString() );
>>>>>>            this.searcher.getIndexReader().directory().close();
>>>>>>            this.searcher.getIndexReader().close();
>>>>>>            this.searcher.close();
>>>>>>        } catch (IOException e) {
>>>>>>            System.out.println(" caught a " + e.getClass() +
"\n with
>>>>>> message: " + e.getMessage());
>>>>>>        }
>>>>>>    }
>>>>>> jboss don't close *.cfs index files, everytime that I make a new
>>>>>> search
>>>>>> a
>>>>>> new file is caught by jboss.
>>>>>> If I make 5 searchs jboss caught 5 *.cfs, I see it with lsof command,
>>>>>> the
>>>>>> only way to free this files and close it is stopping jboss.
>>>>>>
>>>>>> The problem is that after search many times,  jboss go down and
I
>>>>>> have
>>>>>> to
>>>>>> restart it.
>>>>>>
>>>>>> Do you have the same problem with IndexSearch or is only with
>>>>>> IndexWriter?
>>>>>>
>>>>>> Albert Juhe
>>>>>> Learning Technologies
>>>>>> Universitat Oberta de Catalunya
>>>>>>
>>>>>>
>>>>>> Michael McCandless-2 wrote:
>>>>>>>
>>>>>>> Does this look like a real leak John?  You're definitely closing
>>>>>>> every
>>>>>>> reader you get back from getReader?
>>>>>>>
>>>>>>> Mike
>>>>>>>
>>>>>>> On Sun, Nov 8, 2009 at 10:41 PM, John Wang <john.wang@gmail.com>
>>>>>>> wrote:
>>>>>>>> I am seeing the samething, but only when IndexWriter.getReader
is
>>>>>>>> called
>>>>>>>> at
>>>>>>>> a high rate.
>>>>>>>>
>>>>>>>> from lsof, I see file handles growing.
>>>>>>>>
>>>>>>>> -John
>>>>>>>>
>>>>>>>> On Sun, Nov 8, 2009 at 7:29 PM, Daniel Noll <daniel@nuix.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi all.
>>>>>>>>>
>>>>>>>>> We updated to Lucene 2.9, and now we find that after
closing our
>>>>>>>>> text
>>>>>>>>> index, it is not possible to rename the directory in
which it
>>>>>>>>> resides
>>>>>>>>> (we are actually renaming a directory further up the
hierarchy.)
>>>>>>>>>
>>>>>>>>> We discovered that the following files were still open
by the
>>>>>>>>> process:
>>>>>>>>>
>>>>>>>>>  _0.tis, _0.frq, _0.prx, _0.fdt, _0.fdx, _0.tvx, _0.tvd,
_0.tvf
>>>>>>>>>
>>>>>>>>> We are calling IndexWriter.close() shortly before attempting
to
>>>>>>>>> write
>>>>>>>>> to the directory (a few lines of code earlier) so I suspect
it
>>>>>>>>> could
>>>>>>>>> be related to timing somehow if Lucene is perhaps still
doing
>>>>>>>>> something on a background thread at this time (though
I was under
>>>>>>>>> the
>>>>>>>>> impression that close() waited for merges and so forth
to complete
>>>>>>>>> before returning.)
>>>>>>>>>
>>>>>>>>> Daniel
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Daniel Noll                            Forensic
and eDiscovery
>>>>>>>>> Software
>>>>>>>>> Senior Developer                        
     The world's most
>>>>>>>>> advanced
>>>>>>>>> Nuix                                
               email data
>>>>>>>>> analysis
>>>>>>>>> http://nuix.com/                        
       and eDiscovery
>>>>>>>>> software
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>> http://old.nabble.com/IndexWriter.close%28%29-no-longer-seems-to-close-everything-tp26260801p26298910.html
>>>>>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://old.nabble.com/IndexWriter.close%28%29-no-longer-seems-to-close-everything-tp26260801p26299405.html
>>>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/IndexWriter.close%28%29-no-longer-seems-to-close-everything-tp26260801p26299941.html
>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
> 
> 

-- 
View this message in context: http://old.nabble.com/IndexWriter.close%28%29-no-longer-seems-to-close-everything-tp26260801p26334253.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message