cocoon-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vadim Gritsenko" <vadim.gritse...@verizon.net>
Subject RE: Problem with SimpleLuceneCocoonIndexerImpl (Too many open files)
Date Fri, 15 Mar 2002 21:15:17 GMT
> From: leo leonid [mailto:tek@leonid.de]
> 
> On Friday, March 15, 2002, at 08:46  Uhr, Vadim Gritsenko wrote:
> 
> > I guess it is because you are under DOS. You have to add some kind
of
> 
> > FILES=500
> 
> > line to the config.sys.
> 
> > Vadim
> 
> 
> maybe you better go to sleep now...

Nah, I better go home when work is over ;)

Still, what's your OS/file limit?


> and hand this problem over to your coders :-)
> 
> it seems that the resources (open file descriptors) are getting short
> maybe they are never closed. The problem occurs in traversing
> a somehow bigger file tree. I am not sure if the descriptors are lost
> on the files to index or on the traversed directory hierarchy.

Crawler/Indexer accesses Cocoon over java.net, so no files involved
here. Assuming that core Cocoon does not leak files (I never heard about
it), I would suggest you to look into Lucene's IndexWriter - may be it
takes too much files or does not close or looses them.


> code snippet from SimpleLuceneCocoonIndexerImpl.java:
> 
>   try {
>             lxi = (LuceneXMLIndexer)
manager.lookup(LuceneXMLIndexer.ROLE);
> 
>             writer = new IndexWriter(index, analyzer, create);
>             writer.mergeFactor = this.mergeFactor;
> 
>             cocoonCrawler = (CocoonCrawler)
> manager.lookup(CocoonCrawler.ROLE);
>             cocoonCrawler.crawl(base_url);
> 
>             Iterator cocoonCrawlerIterator = cocoonCrawler.iterator();
>             while (cocoonCrawlerIterator.hasNext()) {
>                 URL crawl_url = (URL) cocoonCrawlerIterator.next();

<snip/>

>                 // build lucene documents from the content of the
crawl_url
>                 Iterator i = lxi.build(crawl_url).iterator();
> 
>                 // add all built lucene documents
>                 while (i.hasNext()) {
>                     writer.addDocument((Document) i.next());
>                 }
>             }
>             // optimize it
>             writer.optimize();
>         } catch (IOException ioe) {
>             throw new ProcessingException("IOException in index()",
ioe);
>         } catch (ComponentException ce) {
> [... stuff deleted ...]
> 
> 
> Thx in advance for better help...

Is this one better?

> /leo

Vadim


---------------------------------------------------------------------
Please check that your question has not already been answered in the
FAQ before posting. <http://xml.apache.org/cocoon/faqs.html>

To unsubscribe, e-mail: <cocoon-users-unsubscribe@xml.apache.org>
For additional commands, e-mail: <cocoon-users-help@xml.apache.org>


Mime
View raw message