cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Neumcke <pneum...@virbus.de>
Subject AW: Lucene Problems
Date Mon, 04 Nov 2002 10:25:30 GMT
Hi Jeremy,

we had this problem too (on Linux)- solved it by setting the maximum number
of open file descriptors to 2048:

ulimit -n 2048


> -----Ursprungliche Nachricht-----
> Von: Jeremy Quinn [mailto:jeremy@media.demon.co.uk]
> Gesendet: Montag, 4. November 2002 11:06
> An: cocoon-dev@xml.apache.org
> Betreff: Re: Lucene Problems
> 
> 
> 
> On Sunday, Nov 3, 2002, at 20:34 Europe/London, Bernhard Huber wrote:
> 
> > hi,
> >
> 
> Many thanks for your reply, Bernhard.
> 
> >>  IOException in index()
> 
> >> This causes the index to be incomplete, and causes further 
> problems   
> >> when you try to search using the index.
> >>
> >> If I use the Lucene Samples in Cocoon, and index my Cocoon   
> >> documentation, I always get this error.
> >
> > Hm, i once got this kind of error, too,
> > but i can't remember exactly how i did solve it,
> > perhaps you must set the mergefactor of lucene differently.
> > As much as i remember lucene has some options about how 
> many files are  
> > kept opened in case of indexing,
> > it later merges the indexes files, ....
> 
> Ah Ha! I was beginning to think it was some weakness of 
> MacOSX JVM (or  
> maybe it still is ....).
> 
> Have you any idea where this is configured?
> 
> >> Before I started to have the indexing failures, I had another  
> >> problem,  this time much more difficult to replicate.
> >>
> >> About every 10 searches, I would get a "Bad Resource" error from   
> >> LuceneGenerator. A simple reload of the page would give me 
> the search  
> >>  hits I expected.
> >>
> >> Can anyone think of a Cocoon cause for these problems, or 
> should I  
> >> get  onto the Lucene team about them?
> >
> > Do you have anymore detailed exception stack traces?
> 
> I do for the indexing, but not for the intermittent fault, which has  
> not happened again since the indexing started failing.
> 
> Here is the indexing exception report:
> 
> Cocoon 2 - Internal server error
> 
> type fatal
> 
> message IOException in index()
> 
> description org.apache.cocoon.ProcessingException: IOException in  
> index(): java.io.FileNotFoundException:  
> /Users/jermq/Library/TomCat/work/Standalone/localhost/cocoon/cocoon- 
> files/index/_1d.f30 (Too many open files)
> 
> sender org.apache.cocoon.servlet.CocoonServlet
> 
> source Cocoon servlet
> 
> stack-trace
> 
> org.apache.cocoon.ProcessingException: IOException in index():  
> java.io.FileNotFoundException:  
> /Users/jermq/Library/TomCat/work/Standalone/localhost/cocoon/cocoon- 
> files/index/_1d.f30 (Too many open files)
> 	at  
> org.apache.cocoon.components.search.SimpleLuceneCocoonIndexerI
> mpl.index( 
> SimpleLuceneCocoonIndexerImpl.java:261)
> 	at  
> org.apache.cocoon.www.samples.search.create_index_xsp.createIndex(/ 
> Users/jermq/Library/TomCat/work/Standalone/localhost/cocoon/cocoon- 
> files/org/apache/cocoon/www/samples/search/create_index_xsp.java:106)
> 	at  
> org.apache.cocoon.www.samples.search.create_index_xsp.generate
> (/Users/ 
> jermq/Library/TomCat/work/Standalone/localhost/cocoon/cocoon-f
> iles/org/ 
> apache/cocoon/www/samples/search/create_index_xsp.java:190)
> 	at  
> org.apache.cocoon.generation.ServerPagesGenerator.generate(Ser
> verPagesGe 
> nerator.java:269)
> 	at  
> org.apache.cocoon.components.pipeline.AbstractProcessingPipeli
> ne.process 
> XMLPipeline(AbstractProcessingPipeline.java:512)
> 	at  
> org.apache.cocoon.components.pipeline.impl.AbstractCachingProc
> essingPipe 
> line.processXMLPipeline(AbstractCachingProcessingPipeline.java:204)
> 	at  
> org.apache.cocoon.components.pipeline.AbstractProcessingPipeli
> ne.process 
> (AbstractProcessingPipeline.java:483)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.SerializeNo
> de.invoke( 
> SerializeNode.java:149)
> 	at  
> org.apache.cocoon.components.treeprocessor.AbstractParentProce
> ssingNode. 
> invokeNodes(AbstractParentProcessingNode.java:85)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.PreparableM
> atchNode.i 
> nvoke(PreparableMatchNode.java:166)
> 	at  
> org.apache.cocoon.components.treeprocessor.AbstractParentProce
> ssingNode. 
> invokeNodes(AbstractParentProcessingNode.java:109)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.PipelineNod
> e.invoke(P 
> ipelineNode.java:153)
> 	at  
> org.apache.cocoon.components.treeprocessor.AbstractParentProce
> ssingNode. 
> invokeNodes(AbstractParentProcessingNode.java:109)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNo
> de.invoke( 
> PipelinesNode.java:143)
> 	at  
> org.apache.cocoon.components.treeprocessor.TreeProcessor.proce
> ss(TreePro 
> cessor.java:326)
> 	at  
> org.apache.cocoon.components.treeprocessor.TreeProcessor.proce
> ss(TreePro 
> cessor.java:308)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.MountNode.i
> nvoke(Moun 
> tNode.java:131)
> 	at  
> org.apache.cocoon.components.treeprocessor.AbstractParentProce
> ssingNode. 
> invokeNodes(AbstractParentProcessingNode.java:85)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.PreparableM
> atchNode.i 
> nvoke(PreparableMatchNode.java:166)
> 	at  
> org.apache.cocoon.components.treeprocessor.AbstractParentProce
> ssingNode. 
> invokeNodes(AbstractParentProcessingNode.java:109)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.PipelineNod
> e.invoke(P 
> ipelineNode.java:153)
> 	at  
> org.apache.cocoon.components.treeprocessor.AbstractParentProce
> ssingNode. 
> invokeNodes(AbstractParentProcessingNode.java:109)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNo
> de.invoke( 
> PipelinesNode.java:143)
> 	at  
> org.apache.cocoon.components.treeprocessor.TreeProcessor.proce
> ss(TreePro 
> cessor.java:326)
> 	at  
> org.apache.cocoon.components.treeprocessor.TreeProcessor.proce
> ss(TreePro 
> cessor.java:308)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.MountNode.i
> nvoke(Moun 
> tNode.java:131)
> 	at  
> org.apache.cocoon.components.treeprocessor.AbstractParentProce
> ssingNode. 
> invokeNodes(AbstractParentProcessingNode.java:85)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.PreparableM
> atchNode.i 
> nvoke(PreparableMatchNode.java:166)
> 	at  
> org.apache.cocoon.components.treeprocessor.AbstractParentProce
> ssingNode. 
> invokeNodes(AbstractParentProcessingNode.java:109)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.PipelineNod
> e.invoke(P 
> ipelineNode.java:153)
> 	at  
> org.apache.cocoon.components.treeprocessor.AbstractParentProce
> ssingNode. 
> invokeNodes(AbstractParentProcessingNode.java:109)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNo
> de.invoke( 
> PipelinesNode.java:143)
> 	at  
> org.apache.cocoon.components.treeprocessor.TreeProcessor.proce
> ss(TreePro 
> cessor.java:326)
> 	at  
> org.apache.cocoon.components.treeprocessor.TreeProcessor.proce
> ss(TreePro 
> cessor.java:308)
> 	at org.apache.cocoon.Cocoon.process(Cocoon.java:595)
> 	at  
> org.apache.cocoon.servlet.CocoonServlet.service(CocoonServlet.
> java:1069)
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
> 	at  
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilt
> er(Applica 
> tionFilterChain.java:247)
> 	at  
> org.apache.catalina.core.ApplicationFilterChain.doFilter(Appli
> cationFilt 
> erChain.java:193)
> 	at  
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardW
> rapperValv 
> e.java:260)
> 	at  
> org.apache.catalina.core.StandardPipeline$StandardPipelineValv
> eContext.i 
> nvokeNext(StandardPipeline.java:643)
> 	at  
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipel
> ine.java:4 
> 80)
> 	at  
> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
> 	at  
> org.apache.catalina.core.StandardContextValve.invoke(StandardC
> ontextValv 
> e.java:191)
> 	at  
> org.apache.catalina.core.StandardPipeline$StandardPipelineValv
> eContext.i 
> nvokeNext(StandardPipeline.java:643)
> 	at  
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipel
> ine.java:4 
> 80)
> 	at  
> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
> 	at  
> org.apache.catalina.core.StandardContext.invoke(StandardContex
> t.java:239 
> 6)
> 	at  
> org.apache.catalina.core.StandardHostValve.invoke(StandardHost
> Valve.java 
> :180)
> 	at  
> org.apache.catalina.core.StandardPipeline$StandardPipelineValv
> eContext.i 
> nvokeNext(StandardPipeline.java:643)
> 	at  
> org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDi
> spatcherVa 
> lve.java:170)
> 	at  
> org.apache.catalina.core.StandardPipeline$StandardPipelineValv
> eContext.i 
> nvokeNext(StandardPipeline.java:641)
> 	at  
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReport
> Valve.java 
> :172)
> 	at  
> org.apache.catalina.core.StandardPipeline$StandardPipelineValv
> eContext.i 
> nvokeNext(StandardPipeline.java:641)
> 	at  
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipel
> ine.java:4 
> 80)
> 	at  
> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
> 	at  
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEn
> gineValve. 
> java:174)
> 	at  
> org.apache.catalina.core.StandardPipeline$StandardPipelineValv
> eContext.i 
> nvokeNext(StandardPipeline.java:643)
> 	at  
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipel
> ine.java:4 
> 80)
> 	at  
> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
> 	at  
> org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.
> java:223)
> 	at  
> org.apache.coyote.http11.Http11Processor.process(Http11Process
> or.java:40 
> 5)
> 	at  
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandle
> r.processC 
> onnection(Http11Protocol.java:380)
> 	at  
> org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoi
> nt.java:50 
> 8)
> 	at  
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(
> ThreadPool 
> .java:533)
> 	at java.lang.Thread.run(Thread.java:491)
> java.io.FileNotFoundException:  
> /Users/jermq/Library/TomCat/work/Standalone/localhost/cocoon/cocoon- 
> files/index/_1d.f30 (Too many open files)
> 	at java.io.RandomAccessFile.open(Native Method)
> 	at java.io.RandomAccessFile.(RandomAccessFile.java:93)
> 	at java.io.RandomAccessFile.(RandomAccessFile.java:138)
> 	at 
> org.apache.lucene.store.FSInputStream$Descriptor.(Unknown Source)
> 	at org.apache.lucene.store.FSInputStream.(Unknown Source)
> 	at org.apache.lucene.store.FSDirectory.openFile(Unknown Source)
> 	at 
> org.apache.lucene.index.SegmentReader.openNorms(Unknown Source)
> 	at org.apache.lucene.index.SegmentReader.(Unknown Source)
> 	at 
> org.apache.lucene.index.IndexWriter.mergeSegments(Unknown Source)
> 	at org.apache.lucene.index.IndexWriter.optimize(Unknown Source)
> 	at  
> org.apache.cocoon.components.search.SimpleLuceneCocoonIndexerI
> mpl.index( 
> SimpleLuceneCocoonIndexerImpl.java:259)
> 	at  
> org.apache.cocoon.www.samples.search.create_index_xsp.createIndex(/ 
> Users/jermq/Library/TomCat/work/Standalone/localhost/cocoon/cocoon- 
> files/org/apache/cocoon/www/samples/search/create_index_xsp.java:106)
> 	at  
> org.apache.cocoon.www.samples.search.create_index_xsp.generate
> (/Users/ 
> jermq/Library/TomCat/work/Standalone/localhost/cocoon/cocoon-f
> iles/org/ 
> apache/cocoon/www/samples/search/create_index_xsp.java:190)
> 	at  
> org.apache.cocoon.generation.ServerPagesGenerator.generate(Ser
> verPagesGe 
> nerator.java:269)
> 	at  
> org.apache.cocoon.components.pipeline.AbstractProcessingPipeli
> ne.process 
> XMLPipeline(AbstractProcessingPipeline.java:512)
> 	at  
> org.apache.cocoon.components.pipeline.impl.AbstractCachingProc
> essingPipe 
> line.processXMLPipeline(AbstractCachingProcessingPipeline.java:204)
> 	at  
> org.apache.cocoon.components.pipeline.AbstractProcessingPipeli
> ne.process 
> (AbstractProcessingPipeline.java:483)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.SerializeNo
> de.invoke( 
> SerializeNode.java:149)
> 	at  
> org.apache.cocoon.components.treeprocessor.AbstractParentProce
> ssingNode. 
> invokeNodes(AbstractParentProcessingNode.java:85)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.PreparableM
> atchNode.i 
> nvoke(PreparableMatchNode.java:166)
> 	at  
> org.apache.cocoon.components.treeprocessor.AbstractParentProce
> ssingNode. 
> invokeNodes(AbstractParentProcessingNode.java:109)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.PipelineNod
> e.invoke(P 
> ipelineNode.java:153)
> 	at  
> org.apache.cocoon.components.treeprocessor.AbstractParentProce
> ssingNode. 
> invokeNodes(AbstractParentProcessingNode.java:109)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNo
> de.invoke( 
> PipelinesNode.java:143)
> 	at  
> org.apache.cocoon.components.treeprocessor.TreeProcessor.proce
> ss(TreePro 
> cessor.java:326)
> 	at  
> org.apache.cocoon.components.treeprocessor.TreeProcessor.proce
> ss(TreePro 
> cessor.java:308)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.MountNode.i
> nvoke(Moun 
> tNode.java:131)
> 	at  
> org.apache.cocoon.components.treeprocessor.AbstractParentProce
> ssingNode. 
> invokeNodes(AbstractParentProcessingNode.java:85)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.PreparableM
> atchNode.i 
> nvoke(PreparableMatchNode.java:166)
> 	at  
> org.apache.cocoon.components.treeprocessor.AbstractParentProce
> ssingNode. 
> invokeNodes(AbstractParentProcessingNode.java:109)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.PipelineNod
> e.invoke(P 
> ipelineNode.java:153)
> 	at  
> org.apache.cocoon.components.treeprocessor.AbstractParentProce
> ssingNode. 
> invokeNodes(AbstractParentProcessingNode.java:109)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNo
> de.invoke( 
> PipelinesNode.java:143)
> 	at  
> org.apache.cocoon.components.treeprocessor.TreeProcessor.proce
> ss(TreePro 
> cessor.java:326)
> 	at  
> org.apache.cocoon.components.treeprocessor.TreeProcessor.proce
> ss(TreePro 
> cessor.java:308)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.MountNode.i
> nvoke(Moun 
> tNode.java:131)
> 	at  
> org.apache.cocoon.components.treeprocessor.AbstractParentProce
> ssingNode. 
> invokeNodes(AbstractParentProcessingNode.java:85)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.PreparableM
> atchNode.i 
> nvoke(PreparableMatchNode.java:166)
> 	at  
> org.apache.cocoon.components.treeprocessor.AbstractParentProce
> ssingNode. 
> invokeNodes(AbstractParentProcessingNode.java:109)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.PipelineNod
> e.invoke(P 
> ipelineNode.java:153)
> 	at  
> org.apache.cocoon.components.treeprocessor.AbstractParentProce
> ssingNode. 
> invokeNodes(AbstractParentProcessingNode.java:109)
> 	at  
> org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNo
> de.invoke( 
> PipelinesNode.java:143)
> 	at  
> org.apache.cocoon.components.treeprocessor.TreeProcessor.proce
> ss(TreePro 
> cessor.java:326)
> 	at  
> org.apache.cocoon.components.treeprocessor.TreeProcessor.proce
> ss(TreePro 
> cessor.java:308)
> 	at org.apache.cocoon.Cocoon.process(Cocoon.java:595)
> 	at  
> org.apache.cocoon.servlet.CocoonServlet.service(CocoonServlet.
> java:1069)
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
> 	at  
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilt
> er(Applica 
> tionFilterChain.java:247)
> 	at  
> org.apache.catalina.core.ApplicationFilterChain.doFilter(Appli
> cationFilt 
> erChain.java:193)
> 	at  
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardW
> rapperValv 
> e.java:260)
> 	at  
> org.apache.catalina.core.StandardPipeline$StandardPipelineValv
> eContext.i 
> nvokeNext(StandardPipeline.java:643)
> 	at  
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipel
> ine.java:4 
> 80)
> 	at  
> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
> 	at  
> org.apache.catalina.core.StandardContextValve.invoke(StandardC
> ontextValv 
> e.java:191)
> 	at  
> org.apache.catalina.core.StandardPipeline$StandardPipelineValv
> eContext.i 
> nvokeNext(StandardPipeline.java:643)
> 	at  
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipel
> ine.java:4 
> 80)
> 	at  
> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
> 	at  
> org.apache.catalina.core.StandardContext.invoke(StandardContex
> t.java:239 
> 6)
> 	at  
> org.apache.catalina.core.StandardHostValve.invoke(StandardHost
> Valve.java 
> :180)
> 	at  
> org.apache.catalina.core.StandardPipeline$StandardPipelineValv
> eContext.i 
> nvokeNext(StandardPipeline.java:643)
> 	at  
> org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDi
> spatcherVa 
> lve.java:170)
> 	at  
> org.apache.catalina.core.StandardPipeline$StandardPipelineValv
> eContext.i 
> nvokeNext(StandardPipeline.java:641)
> 	at  
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReport
> Valve.java 
> :172)
> 	at  
> org.apache.catalina.core.StandardPipeline$StandardPipelineValv
> eContext.i 
> nvokeNext(StandardPipeline.java:641)
> 	at  
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipel
> ine.java:4 
> 80)
> 	at  
> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
> 	at  
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEn
> gineValve. 
> java:174)
> 	at  
> org.apache.catalina.core.StandardPipeline$StandardPipelineValv
> eContext.i 
> nvokeNext(StandardPipeline.java:643)
> 	at  
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipel
> ine.java:4 
> 80)
> 	at  
> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
> 	at  
> org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.
> java:223)
> 	at  
> org.apache.coyote.http11.Http11Processor.process(Http11Process
> or.java:40 
> 5)
> 	at  
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandle
> r.processC 
> onnection(Http11Protocol.java:380)
> 	at  
> org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoi
> nt.java:50 
> 8)
> 	at  
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(
> ThreadPool 
> .java:533)
> 	at java.lang.Thread.run(Thread.java:491)
> 
> request-uri
> 
> /cocoon/samples/search/create
> 
> path-info
> 
> samples/search/create
> 
> 
> 
> Thanks again
> 
> regards Jeremy
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
> For additional commands, email: cocoon-dev-help@xml.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org


Mime
View raw message