lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From maureen tanuwidjaja <autumn_musi...@yahoo.com>
Subject Urgent : How much actually the disk space needed to optimize the index?
Date Tue, 13 Mar 2007 08:04:16 GMT

Dear All
  
  How much actually the disk space needed to optimize the index?The  explanation given in
documentation seems to be very different with the  practical situation
  
  I have an index file of size 18.6 G and I am going to optimize it.I  keep this index in
mobile Hard Disk with capacity of 100 Gb....I did  not use any index reader,and I merely call
index writer to optimize  this index.However,to my surprise,now while optimizing, the Index
size  grow to almost occupy all the free space.I am preety sure that later it  will terminated
due to there is no sufficient disk space.
  
  This is the content on the index file
  ------------------------------------------------------------------------------------------
  Microsoft Windows XP [Version 5.1.2600]
  (C) Copyright 1985-2001 Microsoft Corp.
  F:\DI>dir
   Volume in drive F has no label.
   Volume Serial Number is 9454-C24E
  
   Directory of F:\DI
  
  03/13/2007  02:14 PM    <DIR>          .
  03/13/2007  02:14 PM    <DIR>          ..
  03/13/2007  02:14  PM                 20 segments.gen
  03/13/2007  02:14  PM                 67 segments_34s4
  03/13/2007  12:06  PM                  0 write.lock
  03/13/2007  02:14 PM    41,705,009,152 _1ke1.cfs
  03/13/2007  12:15 PM     1,638,320,227 _1ke1.fdt
  03/13/2007  12:15 PM         4,461,912 _1ke1.fdx
  03/13/2007  12:09 PM         6,295,065 _1ke1.fnm
  03/13/2007  12:26 PM       232,520,666 _1ke1.frq
  03/13/2007  02:08 PM    44,927,549,671 _1ke1.nrm
  03/13/2007  12:26 PM       170,766,513 _1ke1.prx
  03/13/2007  12:26 PM         1,281,924 _1ke1.tii
  03/13/2007  12:26 PM       103,094,835 _1ke1.tis
  03/13/2007  02:14 PM        51,688,575 _1ke1.tvd
  03/13/2007  02:14 PM       882,304,866 _1ke1.tvf
  03/13/2007  02:14 PM         4,461,916 _1ke1.tvx
  03/12/2007  03:24 PM     5,594,336,501 _8km.cfs
                16 File(s) 95,322,091,910 bytes
                  2 Dir(s)   3,915,960,320 bytes free
  
  F:\DI>
  
  
  ----------------------------------------------------------------------------------------
  
  
  I wonder what was happening...I read in the documentation that calling  the optimizer will
need available disk space about 2 times current  index size.And I have more than 2 times 18.6
Gb of free space !!!
  
  I really confuse and dont know what is going wrong.This is my code for optimizing :
  ----------------------------------------------------------------------------------------
  package edu.ntu.ce.maureen.index.optimize;
  
  import java.util.Date;
  
  import org.apache.lucene.analysis.Analyzer;
  import org.apache.lucene.analysis.snowball.SnowballAnalyzer;
  import org.apache.lucene.analysis.standard.StandardAnalyzer;
  import org.apache.lucene.index.IndexWriter;
  
  public class OptimizeDI {
  private static Analyzer analyzer = new SnowballAnalyzer("English",StandardAnalyzer.STOP_WORDS);
   
  private static IndexWriter writerOpt;
  
  public static void OpenIndexDir(String indexDir)throws Exception{
      try
      {
          
          writerOpt = new IndexWriter(indexDir,analyzer,false);
      }
      catch (Exception e)
      {
          System.out.println("Cannot create index writer");
          e.printStackTrace();
      }
  }
  
  public static void OptimizeIndex() throws Exception{
      try
      {    
          System.out.println("Optimizing DI...");
          writerOpt.optimize();
          
      }catch(Exception e)
      {
          System.out.println("Exception in writerOpt.optimize()");
          e.printStackTrace();
      }
      
  }
  
  public static void closeIndex() throws Exception{
      try
      {    
          writerOpt.close();
          
      }catch(Exception e)
      {
          System.out.println("Cannot close index writer");
          e.printStackTrace();
      }
  }    
  public static void main(String args[]){
      long start = new Date().getTime();
      
      try{
      OptimizeDI.OpenIndexDir("F:/DI");
      OptimizeDI.OptimizeIndex();
      OptimizeDI.closeIndex();}catch(Exception e){
          System.out.println("Fail to optimize DI");
          e.printStackTrace();
      }
      long end = new Date().getTime();
      System.out.println("Optimized DI is created in "+(end-start)+" ms");
  }
  
  }
  
  ----------------------------------------------------------------------------------------
  
  Can somebody help me?Thanks a lot >_<
  
  
  Regards,
  Maureen
  

 
---------------------------------
Never miss an email again!
Yahoo! Toolbar alerts you the instant new Mail arrives. Check it out.
Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message