lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Terry Steichen" <te...@net-frame.com>
Subject Re: High number of files in the index
Date Mon, 05 May 2003 16:42:55 GMT
Victor,

I believe it's not only the number of fields, but how many fields you select
to have their contents included in the index.  If you don't need the content
accessible via the index, that can make a big difference.

HTH,

Terry

----- Original Message -----
From: "Victor Hadianto" <victorh@nuix.com.au>
To: "Lucene Users List" <lucene-user@jakarta.apache.org>
Sent: Monday, May 05, 2003 2:10 AM
Subject: Re: High number of files in the index


> Yes this seems to be the problem. I had to rewrite the indexer and
indexing it
> a smarter way. Now keeping the number of fields down and the number of
files
> in the Lucene index to a more acceptable level again.
>
> victor
>
> On Mon, 5 May 2003 03:58 pm, Sushma Sinha wrote:
> > The most obvious reason seems to be the increase in number of fields.
> >  I guess lucene creates one file for each field in the index.
> > You can check by looking at the file names in the index, if you have N
> > number of files with the common prefix ,
> > but different suffix , those are all created for different fields.
> >
> > And also , is the lucene performance affected by the no of files in the
> > index. I think u can look at the index size as a whole
> > And if it affects the performance, is there a way to merge the files and
do
> > further optimization?.. as I have not much info about the additional
files
> > created in the index.
> >
> > - Sushma
> >
> >
> > ----- Original Message -----
> > From: "Victor Hadianto" <victorh@nuix.com.au>
> > To: "Lucene Users List" <lucene-user@jakarta.apache.org>
> > Sent: Friday, May 02, 2003 8:57 PM
> > Subject: High number of files in the index
> >
> > > Hi list,
> > >
> > > I'm experiencing a high number of files in the Lucene index, even
after
> > > running optimize I still have over 600 files in my Lucene index. Now
the
> > > scary thing is that's about the same number of document that I
indexed.
> > >
> > > This problem didn't happen before, the only change that I can think of
is
> >
> > that
> >
> > > I'm changing the documents being indexed. Previously all documents
have
> >
> > the
> >
> > > same fields, but now each document has a different set of field
indexed.
> > >
> > > Is this the problem? Will this cause the high number of files in my
index
> > > directory?
> > >
> > > Please someone say no .. because otherwise I'm dead.
> > >
> > >
> > > victor
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
> --
> Victor Hadianto
>
> NUIX Pty Ltd
> Level 8, 143 York Street, Sydney 2000
> Phone: (02) 9283 9010
> Fax:   (02) 9283 9020
>
> This message is intended only for the named recipient. If you are not the
> intended recipient you are notified that disclosing, copying, distributing
> or taking any action in reliance on the contents of this message or
> attachment is strictly prohibited.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message