lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Victor Hadianto <vict...@nuix.com.au>
Subject Re: High number of files in the index
Date Tue, 06 May 2003 02:29:35 GMT
> I believe it's not only the number of fields, but how many fields you
> select to have their contents included in the index.  If you don't need the
> content accessible via the index, that can make a big difference.

Hmm so are you saying even if I have a lot of fields as long as they are not 
stored in the index they are fine? That's strange because all my fields are 
indexed but not stored in the index. Or perhaps I misunderstood what you 
mean?

> Terry

victor

>
> ----- Original Message -----
> From: "Victor Hadianto" <victorh@nuix.com.au>
> To: "Lucene Users List" <lucene-user@jakarta.apache.org>
> Sent: Monday, May 05, 2003 2:10 AM
> Subject: Re: High number of files in the index
>
> > Yes this seems to be the problem. I had to rewrite the indexer and
>
> indexing it
>
> > a smarter way. Now keeping the number of fields down and the number of
>
> files
>
> > in the Lucene index to a more acceptable level again.
> >
> > victor
> >
> > On Mon, 5 May 2003 03:58 pm, Sushma Sinha wrote:
> > > The most obvious reason seems to be the increase in number of fields.
> > >  I guess lucene creates one file for each field in the index.
> > > You can check by looking at the file names in the index, if you have N
> > > number of files with the common prefix ,
> > > but different suffix , those are all created for different fields.
> > >
> > > And also , is the lucene performance affected by the no of files in the
> > > index. I think u can look at the index size as a whole
> > > And if it affects the performance, is there a way to merge the files
> > > and
>
> do
>
> > > further optimization?.. as I have not much info about the additional
>
> files
>
> > > created in the index.
> > >
> > > - Sushma
> > >
> > >
> > > ----- Original Message -----
> > > From: "Victor Hadianto" <victorh@nuix.com.au>
> > > To: "Lucene Users List" <lucene-user@jakarta.apache.org>
> > > Sent: Friday, May 02, 2003 8:57 PM
> > > Subject: High number of files in the index
> > >
> > > > Hi list,
> > > >
> > > > I'm experiencing a high number of files in the Lucene index, even
>
> after
>
> > > > running optimize I still have over 600 files in my Lucene index. Now
>
> the
>
> > > > scary thing is that's about the same number of document that I
>
> indexed.
>
> > > > This problem didn't happen before, the only change that I can think
> > > > of
>
> is
>
> > > that
> > >
> > > > I'm changing the documents being indexed. Previously all documents
>
> have
>
> > > the
> > >
> > > > same fields, but now each document has a different set of field
>
> indexed.
>
> > > > Is this the problem? Will this cause the high number of files in my
>
> index
>
> > > > directory?
> > > >
> > > > Please someone say no .. because otherwise I'm dead.
> > > >
> > > >
> > > > victor
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> >
> > --
> > Victor Hadianto
> >
> > NUIX Pty Ltd
> > Level 8, 143 York Street, Sydney 2000
> > Phone: (02) 9283 9010
> > Fax:   (02) 9283 9020
> >
> > This message is intended only for the named recipient. If you are not the
> > intended recipient you are notified that disclosing, copying,
> > distributing or taking any action in reliance on the contents of this
> > message or attachment is strictly prohibited.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message