Otis
I am using Lucene 1.3 final. Would it help if I move to Lucene 1.4 final?
Rob
----- Original Message -----
From: "Otis Gospodnetic" <otis_gospodnetic@yahoo.com>
To: "Lucene Users List" <lucene-user@jakarta.apache.org>
Sent: Thursday, August 19, 2004 7:13 AM
Subject: Re: Index Size
I thought this was the case. I believe there was a bug in one of the
recent Lucene releases that caused old CFS files not to be removed when
they should be removed. This resulted in your index directory
containing a bunch of old CFS files consuming your disk space.
Try getting a recent nightly build and see if using that takes car eof
your problem.
Otis
--- Rob Jose <rjose89@comcast.net> wrote:
> Hey George
> Thanks for responding. I am using windows and I don't see any hidden
> files.
> I have a ton of CFS files (1366/1405). I have 22 F# (F1, F2, etc.)
> files.
> I have two FDT files and two FDX files. And three FNM files. Add
> these
> files to the deletable and segments file and that is all of the files
> that I
> have. The CFS files are appoximately 11 MB each. The totals I gave
> you
> before were for all of my indexes together. This particular index
> has a
> size of 21.6 GB. The files that it indexed have a size of 89 MB.
>
> OK - I just removed all of the CFS files from the directory and I can
> still
> read my indexes. So know I have to ask what are these CFS files?
> Why are
> they created? And how can I get rid of them if I don't need them. I
> will
> also take a look at the Lucene website to see if I can find any
> information.
>
> Thanks
> Rob
>
> ----- Original Message -----
> From: "Honey George" <honey_george@yahoo.com>
> To: "Lucene Users List" <lucene-user@jakarta.apache.org>
> Sent: Thursday, August 19, 2004 12:29 AM
> Subject: Re: Index Size
>
>
> Hi,
> Please check for hidden files in the index folder. If
> you are using linx, do something like
>
> ls -al <index folder>
>
> I am also facing a similar problem where the index
> size is greater than the data size. In my case there
> were some hidden temproary files which the lucene
> creates.
> That was taking half of the total size.
>
> My problem is that after deleting the temporary files,
> the index size is same as that of the data size. That
> again seems to be a problem. I am yet to find out the
> reason..
>
> Thanks,
> george
>
>
> --- Rob Jose <rjose89@comcast.net> wrote:
> > Hello
> > I have indexed several thousand (52 to be exact)
> > text files and I keep running out of disk space to
> > store the indexes. The size of the documents I have
> > indexed is around 2.5 GB. The size of the Lucene
> > indexes is around 287 GB. Does this seem correct?
> > I am not storing the contents of the file, just
> > indexing and tokenizing. I am using Lucene 1.3
> > final. Can you guys let me know what you are
> > experiencing? I don't want to go into production
> > with something that I should be configuring better.
> >
> >
> > I am not sure if this helps, but I have a temp index
> > and a real index. I index the file into the temp
> > index, and then merge the temp index into the real
> > index using the addIndexes method on the
> > IndexWriter. I have also set the production writer
> > setUseCompoundFile to true. I did not set this on
> > the temp index. The last thing that I do before
> > closing the production writer is to call the
> > optimize method.
> >
> > I would really appreciate any ideas to get the index
> > size smaller if it is at all possible.
> >
> > Thanks
> > Rob
>
>
>
>
>
> ___________________________________________________________ALL-NEW
> Yahoo!
> Messenger - all new features - even more fun!
> http://uk.messenger.yahoo.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
|