lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Britton, Colin" <cbrit...@metatomix.com>
Subject RE: too many open files in system
Date Tue, 09 Apr 2002 16:26:53 GMT
I have worked with the cocoon indexer and it creates a field for each
xml-element and xml-attribute, with complex xml the number of segment
files grows out of control. There is two ways I see to change this.

1) change the cocoon indexer (I looked at this and decided against it)
2) add a stylesheet to the cocoon view to simplify the xml before
indexing (cocoon indexer uses views to extract the links and content for
indexing)

I ended up writing a webapp and indexing the html not the xml because I
also wanted to index jsp's and other things.

Rgds
CB

> -----Original Message-----
> From: Nader S. Henein [mailto:nsh@bayt.net] 
> Sent: Tuesday, April 09, 2002 11:09 AM
> To: Lucene Users List; karl@gan.no; otis_gospodnetic@yahoo.com
> Subject: RE: too many open files in system
> 
> 
> that might be the case I'm indexing 200 000 files each one 
> has about 30 XML fields each one has a set of attributes .. 
> could that be it ?
> 
> -----Original Message-----
> From: Karl Øie [mailto:karl@gan.no]
> Sent: Tuesday, April 09, 2002 7:03 PM
> To: Lucene Users List
> Subject: Re: too many open files in system
> 
> 
> I have worked a little with the cocoon indexer and it indexes 
> each xml-attribute in a Field. I have done some indexing on 
> both plaintext and xml sources and i think the "Too many open 
> files" problem is directly related to number of fields stored 
> in a document in a index.
> 
> the reason for this is that i have never encountered "Too 
> many open files" when indexing clean text into one large 
> field, but when creating many-many fields as required by 
> indexing xml i got a "Too many open files"  until i had to 
> use a ram-dir to index document batches into..
> 
> mvh karl øie
> 
> On Tuesday 09 April 2002 16:42, you wrote:
> > This sounds like a question for Cocoon people, as what you 
> are asking 
> > about seems to be related to Cocoon's usage of Lucene, not the core 
> > Lucene API.
> >
> > Otis
> 
> --
> To unsubscribe, e-mail: 
> <mailto:lucene-user-> unsubscribe@jakarta.apache.org>
> For 
> additional commands, 
> e-mail: <mailto:lucene-user-help@jakarta.apache.org>
> 
> 
> 
> --
> To unsubscribe, e-mail:   
> <mailto:lucene-user-> unsubscribe@jakarta.apache.org>
> For 
> additional commands, 
> e-mail: <mailto:lucene-user-help@jakarta.apache.org>
> 
> 

--
To unsubscribe, e-mail:   <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>


Mime
View raw message