lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Albert Juhe <albertj...@gmail.com>
Subject Re: IndexWriter.close() no longer seems to close everything
Date Fri, 13 Nov 2009 11:46:37 GMT

I don't know how, because the problem is with JBOSS in a productions
environment, in a localhost this don't happens. 

The JBOSS server is in a production environment, contains a lot of projects,
I don't know if Lucene enters in fight with othe libraries.

I don't have control of this computer, I send the war to the production
departament and they install it, I will try to recover more logs files.

Other part of log file:
2009-11-11 16:50:28,567 INFO  [STDOUT]  ************ START SEARCHING
BIBLIOGRAPHY ***************
2009-11-11 16:50:28,577 INFO  [STDOUT] Query
MultiFieldQueryParser:tecnologia 1
2009-11-11 16:50:28,577 INFO  [STDOUT] Query
MultiFieldQueryParser:tecnologia 1
2009-11-11 16:50:28,577 INFO  [STDOUT] Query
MultiFieldQueryParser:tecnologia 1
2009-11-11 16:50:28,731 ERROR
[org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/webapps/uocle].[jsp]]
Servlet.service() for servlet jsp threw exception

The code that generates the error is:

The query is: MultiFieldQueryParser:tecnologia

BooleanQuery bqCodi = new BooleanQuery();

MultiFieldQueryParser mFilefQueryParser = new
MultiFieldQueryParser(this.fields,this.analyzer);            
mFilefQueryParser.setDefaultOperator(QueryParser.AND_OPERATOR); 
queryM  = mFilefQueryParser.parse(this.query_text);
bqCodi.add(queryM, BooleanClause.Occur.SHOULD);

DuplicateFilter df = new DuplicateFilter("title");
df.setKeepMode(DuplicateFilter.KM_USE_FIRST_OCCURRENCE);
this.topDocs = getSearcher().search(bqCodi, df, MAX_TERMES_BIBLIOGRAFIA,
Sort.RELEVANCE);

I don't know if this information can be usefull.

When I changed the Lucene library to 2.4.1 with the same code, only I have
changed how I open Indexreader, in 2.9.1 I opened in readonly=true, the
applications runs perfect.


Michael McCandless-2 wrote:
> 
> Any luck narrowing this to a standalone test case that shows the problem?
> 
> That new exception appears to be inside the Java code created by the
> app server compiling your JSP -- it's not very helpful since it
> doesn't "enter" Lucene.  Can you try to narrow this to a standalone
> test case, too?
> 
> Thanks.
> 
> Mike
> 
> On Fri, Nov 13, 2009 at 5:12 AM, Albert Juhe <albertjuhe@gmail.com> wrote:
>>
>> Hi,
>>
>> About this problem I did a test yesterday, I did a downgrade, I changed
>> versión 2.9.1 to 2.4.1, and the problem has been solved, all the files
>> are
>> closed corretly and JBOSS isn't unstable.
>>
>> Another problem that we have observed is:
>>
>> Sometimes, random success, when you try to make a serach the first time
>> don't return result but the second time yes (Lucene 2.9.1. BooleanQuery).
>> I get this error message:
>> java.lang.NullPointerException
>>        at
>> org.apache.jsp.objects.buscar_005fresposta_jsp._jspService(buscar_005fresposta_jsp.java:631)
>>        at
>> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
>>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
>>        at
>> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:334)
>>        at
>> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:314)
>>        at
>> org.apache.jasper.servlet.JspServlet.service(JspServlet.java:264)
>>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
>>        at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
>>        at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
>>        at
>> org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
>>        at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
>>        at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
>>        at
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
>>        at
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
>>        at
>> org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:175)
>>        at
>> org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:74)
>>        at
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
>>        at
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
>>        at
>> org.jboss.web.tomcat.tc5.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:156)
>>        at
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
>>        at
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
>>        at
>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
>>        at
>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
>>        at
>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
>>        at
>> org.apache.tomcat.util.net.MasterSlaveWorkerThread.run(MasterSlaveWorkerThread.java:112)
>>        at java.lang.Thread.run(Thread.java:595)
>>
>> I don't know whats the meanig of it, but buscar_resposta.jsp is the jsp
>> that
>> makes a Search.
>>
>> With 2.4.1 this problem don't happens.
>>
>>
>> Jason Rutherglen-2 wrote:
>>>
>>> If there's a bug you're seeing, it's helpful to open an issue and post
>>> code reproducing it.
>>>
>>> On Wed, Nov 11, 2009 at 3:41 AM, Albert Juhe <albertjuhe@gmail.com>
>>> wrote:
>>>>
>>>> I think that this is the best way to proceed.
>>>>
>>>> thank you Mike
>>>>
>>>>
>>>>
>>>> Michael McCandless-2 wrote:
>>>>>
>>>>> Can you narrow the leak down to a small self-contained test?
>>>>>
>>>>> Mike
>>>>>
>>>>> On Wed, Nov 11, 2009 at 5:54 AM, Albert Juhe <albertjuhe@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> I don't get any exception.
>>>>>>
>>>>>> thank you Mike
>>>>>>
>>>>>>
>>>>>> Michael McCandless-2 wrote:
>>>>>>>
>>>>>>> Do you see your exception handler printing anything out?
>>>>>>>
>>>>>>> You don't need to close the underlying IndexReader, just the
>>>>>>> IndexSearcher (which will close the IndexReader, since it was
the
>>>>>>> one
>>>>>>> that had opened it).
>>>>>>>
>>>>>>> Mike
>>>>>>>
>>>>>>> On Wed, Nov 11, 2009 at 5:10 AM, Albert Juhe <albertjuhe@gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> I don't know if it's the same problem but I think it's similar,
>>>>>>>>
>>>>>>>> My problem is with the Indexsearcher. I've installed a web
search
>>>>>>>> engine
>>>>>>>> that uses Lucene, after a search I make a close operation
like
>>>>>>>> this:
>>>>>>>>
>>>>>>>> private IndexSearcher searcher;
>>>>>>>>
>>>>>>>> NIOFSDirectory directory = new NIOFSDirectory(new File(path));
>>>>>>>> this.searcher = new IndexSearcher(directory, true);
>>>>>>>>
>>>>>>>>  public void close() {
>>>>>>>>        try {
>>>>>>>>            System.out.println("Closing: "
>>>>>>>> +this.searcher.getIndexReader().directory().toString() );
>>>>>>>>            this.searcher.getIndexReader().directory().close();
>>>>>>>>            this.searcher.getIndexReader().close();
>>>>>>>>            this.searcher.close();
>>>>>>>>        } catch (IOException e) {
>>>>>>>>            System.out.println(" caught a " + e.getClass()
+ "\n
>>>>>>>> with
>>>>>>>> message: " + e.getMessage());
>>>>>>>>        }
>>>>>>>>    }
>>>>>>>> jboss don't close *.cfs index files, everytime that I make
a new
>>>>>>>> search
>>>>>>>> a
>>>>>>>> new file is caught by jboss.
>>>>>>>> If I make 5 searchs jboss caught 5 *.cfs, I see it with lsof
>>>>>>>> command,
>>>>>>>> the
>>>>>>>> only way to free this files and close it is stopping jboss.
>>>>>>>>
>>>>>>>> The problem is that after search many times,  jboss go down
and I
>>>>>>>> have
>>>>>>>> to
>>>>>>>> restart it.
>>>>>>>>
>>>>>>>> Do you have the same problem with IndexSearch or is only
with
>>>>>>>> IndexWriter?
>>>>>>>>
>>>>>>>> Albert Juhe
>>>>>>>> Learning Technologies
>>>>>>>> Universitat Oberta de Catalunya
>>>>>>>>
>>>>>>>>
>>>>>>>> Michael McCandless-2 wrote:
>>>>>>>>>
>>>>>>>>> Does this look like a real leak John?  You're definitely
closing
>>>>>>>>> every
>>>>>>>>> reader you get back from getReader?
>>>>>>>>>
>>>>>>>>> Mike
>>>>>>>>>
>>>>>>>>> On Sun, Nov 8, 2009 at 10:41 PM, John Wang <john.wang@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>> I am seeing the samething, but only when IndexWriter.getReader
is
>>>>>>>>>> called
>>>>>>>>>> at
>>>>>>>>>> a high rate.
>>>>>>>>>>
>>>>>>>>>> from lsof, I see file handles growing.
>>>>>>>>>>
>>>>>>>>>> -John
>>>>>>>>>>
>>>>>>>>>> On Sun, Nov 8, 2009 at 7:29 PM, Daniel Noll <daniel@nuix.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi all.
>>>>>>>>>>>
>>>>>>>>>>> We updated to Lucene 2.9, and now we find that
after closing our
>>>>>>>>>>> text
>>>>>>>>>>> index, it is not possible to rename the directory
in which it
>>>>>>>>>>> resides
>>>>>>>>>>> (we are actually renaming a directory further
up the hierarchy.)
>>>>>>>>>>>
>>>>>>>>>>> We discovered that the following files were still
open by the
>>>>>>>>>>> process:
>>>>>>>>>>>
>>>>>>>>>>>  _0.tis, _0.frq, _0.prx, _0.fdt, _0.fdx, _0.tvx,
_0.tvd, _0.tvf
>>>>>>>>>>>
>>>>>>>>>>> We are calling IndexWriter.close() shortly before
attempting to
>>>>>>>>>>> write
>>>>>>>>>>> to the directory (a few lines of code earlier)
so I suspect it
>>>>>>>>>>> could
>>>>>>>>>>> be related to timing somehow if Lucene is perhaps
still doing
>>>>>>>>>>> something on a background thread at this time
(though I was
>>>>>>>>>>> under
>>>>>>>>>>> the
>>>>>>>>>>> impression that close() waited for merges and
so forth to
>>>>>>>>>>> complete
>>>>>>>>>>> before returning.)
>>>>>>>>>>>
>>>>>>>>>>> Daniel
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Daniel Noll                      
     Forensic and eDiscovery
>>>>>>>>>>> Software
>>>>>>>>>>> Senior Developer                    
         The world's most
>>>>>>>>>>> advanced
>>>>>>>>>>> Nuix                            
                   email data
>>>>>>>>>>> analysis
>>>>>>>>>>> http://nuix.com/                    
           and eDiscovery
>>>>>>>>>>> software
>>>>>>>>>>>
>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>> java-user-help@lucene.apache.org
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> View this message in context:
>>>>>>>> http://old.nabble.com/IndexWriter.close%28%29-no-longer-seems-to-close-everything-tp26260801p26298910.html
>>>>>>>> Sent from the Lucene - Java Users mailing list archive at
>>>>>>>> Nabble.com.
>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>> http://old.nabble.com/IndexWriter.close%28%29-no-longer-seems-to-close-everything-tp26260801p26299405.html
>>>>>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://old.nabble.com/IndexWriter.close%28%29-no-longer-seems-to-close-everything-tp26260801p26299941.html
>>>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/IndexWriter.close%28%29-no-longer-seems-to-close-everything-tp26260801p26334253.html
>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
> 
> 

-- 
View this message in context: http://old.nabble.com/IndexWriter.close%28%29-no-longer-seems-to-close-everything-tp26260801p26335402.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message