Mailing-List: contact lucene-user-help@jakarta.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Lucene Users List" <lucene-user@jakarta.apache.org>
Message-ID: <194001c313c9$e36e8090$0501a8c0@POWERPACK>
From: "Terry Steichen" <terry@net-frame.com>
To: "Lucene Users List" <lucene-user@jakarta.apache.org>
References: <200305030127.15302@bah> <200305051610.31421@bah>
 <17f401c31325$61760cb0$0501a8c0@POWERPACK> <200305061229.35426@bah>
Subject: Re: High number of files in the index
Date: Tue, 6 May 2003 08:20:31 -0400
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

What I said (or meant to say) is that the size of the index depends on not
just the number of fields, but also on how many of the fields have their
contents stored in the index.  (If, as you state, you don't store any field
contents in the index, this is not an issue with you.)

Terry

----- Original Message -----
From: "Victor Hadianto" <victorh@nuix.com.au>
To: "Lucene Users List" <lucene-user@jakarta.apache.org>
Sent: Monday, May 05, 2003 10:29 PM
Subject: Re: High number of files in the index


> > I believe it's not only the number of fields, but how many fields you
> > select to have their contents included in the index.  If you don't need
the
> > content accessible via the index, that can make a big difference.
>
> Hmm so are you saying even if I have a lot of fields as long as they are
not
> stored in the index they are fine? That's strange because all my fields
are
> indexed but not stored in the index. Or perhaps I misunderstood what you
> mean?
>
> > Terry
>
> victor
>
> >
> > ----- Original Message -----
> > From: "Victor Hadianto" <victorh@nuix.com.au>
> > To: "Lucene Users List" <lucene-user@jakarta.apache.org>
> > Sent: Monday, May 05, 2003 2:10 AM
> > Subject: Re: High number of files in the index
> >
> > > Yes this seems to be the problem. I had to rewrite the indexer and
> >
> > indexing it
> >
> > > a smarter way. Now keeping the number of fields down and the number of
> >
> > files
> >
> > > in the Lucene index to a more acceptable level again.
> > >
> > > victor
> > >
> > > On Mon, 5 May 2003 03:58 pm, Sushma Sinha wrote:
> > > > The most obvious reason seems to be the increase in number of
fields.
> > > >  I guess lucene creates one file for each field in the index.
> > > > You can check by looking at the file names in the index, if you have
N
> > > > number of files with the common prefix ,
> > > > but different suffix , those are all created for different fields.
> > > >
> > > > And also , is the lucene performance affected by the no of files in
the
> > > > index. I think u can look at the index size as a whole
> > > > And if it affects the performance, is there a way to merge the files
> > > > and
> >
> > do
> >
> > > > further optimization?.. as I have not much info about the additional
> >
> > files
> >
> > > > created in the index.
> > > >
> > > > - Sushma
> > > >
> > > >
> > > > ----- Original Message -----
> > > > From: "Victor Hadianto" <victorh@nuix.com.au>
> > > > To: "Lucene Users List" <lucene-user@jakarta.apache.org>
> > > > Sent: Friday, May 02, 2003 8:57 PM
> > > > Subject: High number of files in the index
> > > >
> > > > > Hi list,
> > > > >
> > > > > I'm experiencing a high number of files in the Lucene index, even
> >
> > after
> >
> > > > > running optimize I still have over 600 files in my Lucene index.
Now
> >
> > the
> >
> > > > > scary thing is that's about the same number of document that I
> >
> > indexed.
> >
> > > > > This problem didn't happen before, the only change that I can
think
> > > > > of
> >
> > is
> >
> > > > that
> > > >
> > > > > I'm changing the documents being indexed. Previously all documents
> >
> > have
> >
> > > > the
> > > >
> > > > > same fields, but now each document has a different set of field
> >
> > indexed.
> >
> > > > > Is this the problem? Will this cause the high number of files in
my
> >
> > index
> >
> > > > > directory?
> > > > >
> > > > > Please someone say no .. because otherwise I'm dead.
> > > > >
> > > > >
> > > > > victor
> > > > >
> > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> > > > > For additional commands, e-mail:
lucene-user-help@jakarta.apache.org
> > > >
> > >
> ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> > >
> > > --
> > > Victor Hadianto
> > >
> > > NUIX Pty Ltd
> > > Level 8, 143 York Street, Sydney 2000
> > > Phone: (02) 9283 9010
> > > Fax:   (02) 9283 9020
> > >
> > > This message is intended only for the named recipient. If you are not
the
> > > intended recipient you are notified that disclosing, copying,
> > > distributing or taking any action in reliance on the contents of this
> > > message or attachment is strictly prohibited.
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org