lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nrupal Akolkar <nrupalakol...@gmail.com>
Subject Re: Performance Test Failing
Date Fri, 02 Sep 2005 07:01:52 GMT
Actually your problem is that of system virtual memory utilization. The 
thing arrived due to lack of system memory to continue indexing and reading 
the files. SystemErr is the error generated. 
The way to troubleshoot this thing from happening is to create a cluster of 
indexes and then indexing the indexes. 
To create cluster of indexes, use lucene indexing and limit index creation 
by time - i.e. limit indexing by time. And get the last file indexed name. 
Generate a unix script or java monitor program that will start indexing from 
the last file to further in the directory structure. Continue looping till 
you reach end of directory you need indexing. I think this will solve your 
problem.
Nrupal
0091-9879021334

 On 9/2/05, jaina (sent by Nabble.com <http://Nabble.com>) <lists@nabble.com>

wrote: 
> 
> 
> Hi,
> We are running into performace problem as we use lucene. Everything worked 
> fine till yesterday and today when we starte performance testing we saw an 
> error. Here are the details of the error
> 
> 54eda4e1 SystemErr R From Faceted SearchImpl rlinksIndexReader.open(dir) 
> failed
> [9/1/05 17:42:05:646 GMT] 54eda4e1 SystemErr R 
> java.io.FileNotFoundException: 
> /gsa/torgsa/.projects/p1/gsa_cdt_ods/projects/w3perf/datapersist/sales/support/skp/production/index/relatedlinks/secondary/_0.f20

> (Too many open files)
> [9/1/05 17:42:05:647 GMT] 54eda4e1 SystemErr R at 
> java.io.RandomAccessFile.open(Native Method)
> [9/1/05 17:42:05:647 GMT] 54eda4e1 SystemErr R at java.io.RandomAccessFile. 
> (RandomAccessFile.java(Compiled Code))
> [9/1/05 17:42:05:647 GMT] 54eda4e1 SystemErr R at 
> org.apache.lucene.store.FSInputStream$Descriptor. (FSDirectory.java(Inlined 
> Compiled Code))
> [9/1/05 17:42:05:647 GMT] 54eda4e1 SystemErr R at 
> org.apache.lucene.store.FSInputStream. (FSDirectory.java(Inlined Compiled 
> Code))
> [9/1/05 17:42:05:648 GMT] 54eda4e1 SystemErr R at 
> org.apache.lucene.store.FSDirectory.openFile(FSDirectory.java(Compiled 
> Code))
> [9/1/05 17:42:05:648 GMT] 54eda4e1 SystemErr R at 
> org.apache.lucene.index.SegmentReader.openNorms(SegmentReader.java(Compiled 
> Code))
> [9/1/05 17:42:05:648 GMT] 54eda4e1 SystemErr R at 
> org.apache.lucene.index.SegmentReader. (SegmentReader.java:153)
> [9/1/05 17:42:05:648 GMT] 54eda4e1 SystemErr R at 
> org.apache.lucene.index.SegmentReader. (SegmentReader.java:122)
> [9/1/05 17:42:05:648 GMT] 54eda4e1 SystemErr R at 
> org.apache.lucene.index.IndexReader$1.doBody(IndexReader.java:118)
> [9/1/05 17:42:05:649 GMT] 54eda4e1 SystemErr R at 
> org.apache.lucene.store.Lock$With.run(Lock.java(Compiled Code))
> [9/1/05 17:42:05:649 GMT] 54eda4e1 SystemErr R at 
> org.apache.lucene.index.IndexReader.open(IndexReader.java(Compiled Code))
> [9/1/05 17:42:05:649 GMT] 54eda4e1 SystemErr R at 
> com.ibm.sales.support.skp.fsearch.search.FacetedSearchImpl.openRelatedLinksIndex
> (FacetedSearchImpl.java:1603)
> [9/1/05 17:42:05:649 GMT] 54eda4e1 SystemErr R at 
> com.ibm.sales.support.skp.fsearch.search.FacetedSearchImpl.getMatchingRelatedLinksDocs
> (FacetedSearchImpl.java:1071)
> [9/1/05 17:42:05:649 GMT] 54eda4e1 SystemErr R at 
> com.ibm.sales.support.skp.fsearch.search.FacetedSearchImpl.rlSearch(
> FacetedSearchImpl.java(Inlined Compiled Code))
> [9/1/05 17:42:05:650 GMT] 54eda4e1 SystemErr R at 
> com.ibm.sales.support.skp.fsearch.servicehandlers.SearchResultServiceHandler.processRequest
> (SearchResultServiceHandler.java(Compiled Code))
> [9/1/05 17:42:05:650 GMT] 54eda4e1 SystemErr R at 
> com.ibm.sales.support.skp.fsearch.servlets.SearchResultServlet.performTask
> (SearchResultServlet.java:144)
> [9/1/05 17:42:05:650 GMT] 54eda4e1 SystemErr R at 
> com.ibm.sales.support.skp.fsearch.servlets.SearchResultServlet.doGet(
> SearchResultServlet.java:75)
> [9/1/05 17:42:05:650 GMT] 54eda4e1 SystemErr R at 
> javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
> [9/1/05 17:42:05:650 GMT] 54eda4e1 SystemErr R at 
> javax.servlet.http.HttpServlet.service(HttpServlet.java(Compiled Code))
> [9/1/05 17:42:05:651 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.webcontainer.servlet.StrictServletInstance.doService(
> StrictServletInstance.java:110)
> [9/1/05 17:42:05:651 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.webcontainer.servlet.StrictLifecycleServlet._service(
> StrictLifecycleServlet.java:174)
> [9/1/05 17:42:05:651 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.webcontainer.servlet.IdleServletState.service(
> StrictLifecycleServlet.java:313)
> [9/1/05 17:42:05:651 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.webcontainer.servlet.StrictLifecycleServlet.service(
> StrictLifecycleServlet.java:116)
> [9/1/05 17:42:05:651 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.webcontainer.servlet.ServletInstance.service(
> ServletInstance.java:283)
> [9/1/05 17:42:05:652 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.webcontainer.servlet.ValidServletReferenceState.dispatch(
> ValidServletReferenceState.java:42)
> [9/1/05 17:42:05:652 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.webcontainer.servlet.ServletInstanceReference.dispatch(
> ServletInstanceReference.java:40)
> [9/1/05 17:42:05:652 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher.handleWebAppDispatch
> (WebAppRequestDispatcher.java:983)
> [9/1/05 17:42:05:652 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher.dispatch(
> WebAppRequestDispatcher.java:564)
> [9/1/05 17:42:05:652 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher.forward(
> WebAppRequestDispatcher.java:200)
> [9/1/05 17:42:05:653 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.webcontainer.srt.WebAppInvoker.doForward(WebAppInvoker.java
> :119)
> [9/1/05 17:42:05:653 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.webcontainer.srt.WebAppInvoker.handleInvocationHook(
> WebAppInvoker.java:276)
> [9/1/05 17:42:05:653 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.webcontainer.cache.invocation.CachedInvocation.handleInvocation
> (CachedInvocation.java:71)
> [9/1/05 17:42:05:653 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.webcontainer.cache.invocation.CacheableInvocationContext.invoke
> (CacheableInvocationContext.java:116)
> [9/1/05 17:42:05:653 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.webcontainer.srp.ServletRequestProcessor.dispatchByURI(
> ServletRequestProcessor.java:186)
> [9/1/05 17:42:05:654 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.webcontainer.oselistener.OSEListenerDispatcher.service(
> OSEListener.java:334)
> [9/1/05 17:42:05:654 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.webcontainer.http.HttpConnection.handleRequest(
> HttpConnection.java:56)
> [9/1/05 17:42:05:654 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.http.HttpConnection.readAndHandleRequest(HttpConnection.java
> :618)
> [9/1/05 17:42:05:654 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.http.HttpConnection.run(HttpConnection.java:443)
> [9/1/05 17:42:05:654 GMT] 54eda4e1 SystemErr R at 
> com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:672)
> 
> Can someobody help with this?
> 
> Thanks.
> --
> Sent from the Lucene - Java Developer forum at Nabble.com<http://Nabble.com>
> :
> http://www.nabble.com/Performance-Test-Failing-t271579.html#a764224
> 
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message