lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Damien McCarthy" <damien.mccar...@propylon.com>
Subject RE: Optimizing Index
Date Fri, 23 Feb 2007 02:10:45 GMT
What file system is the hard disc? If it is FAT32 one of your indexing files
is probably getting bigger than 4.7 gigs - the maximum file size in FAT32

Damien

-----Original Message-----
From: maureen tanuwidjaja [mailto:autumn_musique@yahoo.com] 
Sent: 23 February 2007 02:07
To: java-user@lucene.apache.org
Subject: Re: Optimizing Index


yes I do have around 75 GB of free space on that HDD...I do not invoke any
index reader...hence the program only calls indexwriter to optimize the
index,and that's it..
   
  I am also perplexed why it tells that it have not enough disk space to do
optimization...
  
Michael McCandless <lucene@mikemccandless.com> wrote:
  
"maureen tanuwidjaja" wrote:

> I had an exsisting index file with the size 20.6 GB...I havent done any
> optimization in this index yet.Now I had a HDD of 100 GB,but apparently
> when I create program to optimize(which simply calls writer.optimize()
> to this indexfile),it gives the error that there is not enough space on
> the disk.
> 
> I read that the size needed to optimize the index is twice as the
> original index size...then it should be around 40 GB instead...I
> confuse why the size of 100 GB is insufficient to do the
> optimization...

Does your disk only have this index? Ie 100 GB - 20.6 = 79.4 GB of
free space?

Do you have reader(s) open on the index when you kick off the
optimize? If so then the temporary free space required is 2X the size
of the index (41.2 GB in your case).

Worse, if your readers are refreshing during the optimize (eg because
on IndexReader.isCurrent() returned false) even more temporary disk
space can be tied up. This is a case LUCENE-710 aims to fix (adding
"commit only on close" to IndexWriter). The workaround for now is to
create app level logic to ensure readers never refresh during
optimize.

If you don't have readers then you really should only need 1X
additional free space at the start of optimize (20.6 GB free in your
case) so I'm baffled why 79.4 GB would not be enough.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org



 
---------------------------------
Now that's room service! Choose from over 150,000 hotels 
in 45,000 destinations on Yahoo! Travel to find your fit.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message