lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shlomit Rosen <SHLOM...@il.ibm.com>
Subject Clean up unused segments
Date Fri, 03 Oct 2014 18:53:07 GMT
Hello, 

We are using lucene 3.6.0. 
We ran optimize on a large collection (250 GB before optimization), during 
which we ran out of disk space. 
After adding more disk space, we re-ran the optimization, and the process 
completed successfully.
However, the index size didn't change very much, and looking at the 
existing segments - it seems that the old files are still there: 
[Notice the optimization ran on Sep 23, but there are files from May]

total 250G
-rw-r----- 1 prodedm edm 907M Jun  6 08:05 _16mx.fdt
-rw-r----- 1 prodedm edm  13M Jun  6 08:05 _16mx.fdx
-rw-r----- 1 prodedm edm  154 Jun  6 08:05 _16mx.fnm
-rw-r----- 1 prodedm edm 2.2G Jun  6 08:41 _16mx.frq
-rw-r----- 1 prodedm edm  13M Jun  6 08:41 _16mx.nrm
-rw-r----- 1 prodedm edm  23G Jun  6 08:41 _16mx.prx
-rw-r----- 1 prodedm edm  18M Jun  6 08:41 _16mx.tii
-rw-r----- 1 prodedm edm 1.7G Jun  6 08:41 _16mx.tis
-rw-r----- 1 prodedm edm   44 Jun  6 22:34 _16mx_5.del
-rw-r----- 1 prodedm edm 5.2G Jun  8 22:32 _186v.cfs
-rw-r----- 1 prodedm edm  636 Jun  8 22:32 _186v_1.del
-rw-r----- 1 prodedm edm 195M May  6 15:09 _1a1.fdt
-rw-r----- 1 prodedm edm 2.6M May  6 15:09 _1a1.fdx
-rw-r----- 1 prodedm edm  154 May  6 15:09 _1a1.fnm
-rw-r----- 1 prodedm edm 573M May  6 15:15 _1a1.frq
-rw-r----- 1 prodedm edm 2.6M May  6 15:15 _1a1.nrm
-rw-r----- 1 prodedm edm 6.3G May  6 15:15 _1a1.prx
-rw-r----- 1 prodedm edm 6.1M May  6 15:15 _1a1.tii
-rw-r----- 1 prodedm edm 589M May  6 15:15 _1a1.tis
-rw-r----- 1 prodedm edm   78 May  7 13:30 _1a1_7.del
-rw-r----- 1 prodedm edm  11G Jun 10 11:25 _1anj.cfs
-rw-r----- 1 prodedm edm   35 Jun 22 15:09 _1anj_4.del
-rw-r----- 1 prodedm edm  12G Jun 12 10:58 _1do2.cfs
-rw-r----- 1 prodedm edm   70 Jun 13 07:10 _1do2_2.del
-rw-r----- 1 prodedm edm  18G Jun 17 02:59 _1i65.cfs
-rw-r----- 1 prodedm edm  222 Jul  6 03:41 _1i65_6.del
-rw-r----- 1 prodedm edm 4.2G Jun 17 14:21 _1j5h.cfs
-rw-r----- 1 prodedm edm  151 Jun 18 07:40 _1j5h_7.del
-rw-r----- 1 prodedm edm 560M Sep 23 20:22 _1o1h.cfs
-rw-r----- 1 prodedm edm 7.0G Sep 23 20:51 _1o1i.cfs
-rw-r----- 1 prodedm edm  12G Sep 23 21:05 _1o1j.cfs
-rw-r----- 1 prodedm edm 9.5G Sep 23 20:57 _1o1k.cfs
-rw-r----- 1 prodedm edm 9.6G Sep 23 20:59 _1o1l.cfs
-rw-r----- 1 prodedm edm 3.1G Sep 23 20:29 _1o1m.cfs
-rw-r----- 1 prodedm edm  22G Sep 23 21:27 _1o1n.cfs
-rw-r----- 1 prodedm edm 4.6G Sep 23 20:36 _1o1o.cfs
-rw-r----- 1 prodedm edm  16G Sep 23 21:10 _1o1p.cfs
-rw-r----- 1 prodedm edm 3.8G Sep 23 20:32 _1o1q.cfs
-rw-r----- 1 prodedm edm 6.4G Sep 23 20:42 _1o1r.cfs
-rw-r----- 1 prodedm edm  17G Sep 23 20:48 _1o1s.cfs
-rw-r----- 1 prodedm edm 2.4G Sep 23 20:32 _1o1t.cfs
-rw-r----- 1 prodedm edm 103M May  7 09:28 _2lz.fdt
-rw-r----- 1 prodedm edm 1.4M May  7 09:28 _2lz.fdx
-rw-r----- 1 prodedm edm  154 May  7 09:28 _2lz.fnm
-rw-r----- 1 prodedm edm 300M May  7 09:33 _2lz.frq
-rw-r----- 1 prodedm edm 1.4M May  7 09:33 _2lz.nrm
-rw-r----- 1 prodedm edm 4.0G May  7 09:33 _2lz.prx
-rw-r----- 1 prodedm edm 2.6M May  7 09:33 _2lz.tii
-rw-r----- 1 prodedm edm 235M May  7 09:33 _2lz.tis
-rw-r----- 1 prodedm edm  22K May  8 06:19 _2lz_1i.del
-rw-r----- 1 prodedm edm 130M May  9 13:11 _6n0.fdt
-rw-r----- 1 prodedm edm 1.7M May  9 13:11 _6n0.fdx
-rw-r----- 1 prodedm edm  154 May  9 13:11 _6n0.fnm
-rw-r----- 1 prodedm edm 342M May  9 13:19 _6n0.frq
-rw-r----- 1 prodedm edm 1.7M May  9 13:19 _6n0.nrm
-rw-r----- 1 prodedm edm 3.9G May  9 13:19 _6n0.prx
-rw-r----- 1 prodedm edm 3.4M May  9 13:19 _6n0.tii
-rw-r----- 1 prodedm edm 322M May  9 13:19 _6n0.tis
-rw-r----- 1 prodedm edm  944 May 10 11:50 _6n0_8.del
-rw-r----- 1 prodedm edm 120M May 12 11:57 _8ry.fdt
-rw-r----- 1 prodedm edm 1.6M May 12 11:57 _8ry.fdx
-rw-r----- 1 prodedm edm  154 May 12 11:57 _8ry.fnm
-rw-r----- 1 prodedm edm 310M May 12 12:04 _8ry.frq
-rw-r----- 1 prodedm edm 1.6M May 12 12:04 _8ry.nrm
-rw-r----- 1 prodedm edm 4.2G May 12 12:04 _8ry.prx
-rw-r----- 1 prodedm edm 3.1M May 12 12:04 _8ry.tii
-rw-r----- 1 prodedm edm 285M May 12 12:04 _8ry.tis
-rw-r----- 1 prodedm edm  467 May 13 06:56 _8ry_h.del
-rw-r----- 1 prodedm edm 4.4G May 13 02:58 _9ls.cfs
-rw-r----- 1 prodedm edm   92 May 13 16:55 _9ls_6.del
-rw-r----- 1 prodedm edm 173M May 14 03:31 _be6.fdt
-rw-r----- 1 prodedm edm 2.3M May 14 03:31 _be6.fdx
-rw-r----- 1 prodedm edm  154 May 14 03:31 _be6.fnm
-rw-r----- 1 prodedm edm 443M May 14 03:38 _be6.frq
-rw-r----- 1 prodedm edm 2.3M May 14 03:38 _be6.nrm
-rw-r----- 1 prodedm edm 5.1G May 14 03:38 _be6.prx
-rw-r----- 1 prodedm edm 4.5M May 14 03:38 _be6.tii
-rw-r----- 1 prodedm edm 416M May 14 03:38 _be6.tis
-rw-r----- 1 prodedm edm   63 May 15 03:05 _be6_6.del
-rw-r----- 1 prodedm edm 5.5G May 16 09:26 _fcd.cfs
-rw-r----- 1 prodedm edm  797 May 17 08:38 _fcd_5.del
-rw-r----- 1 prodedm edm 4.9G May 23 01:45 _n3v.cfs
-rw-r----- 1 prodedm edm   94 May 23 22:45 _n3v_c.del
-rw-r----- 1 prodedm edm 4.3G May 23 14:49 _o56.cfs
-rw-r----- 1 prodedm edm  132 May 24 09:58 _o56_8.del
-rw-r----- 1 prodedm edm 4.5G May 27 05:55 _qn1.cfs
-rw-r----- 1 prodedm edm   27 May 28 03:55 _qn1_3.del
-rw-r----- 1 prodedm edm 5.1G May 29 14:24 _uw3.cfs
-rw-r----- 1 prodedm edm  122 May 30 07:39 _uw3_9.del
-rw-r----- 1 prodedm edm 5.4G May 30 11:38 _woy.cfs
-rw-r----- 1 prodedm edm   61 May 31 10:09 _woy_5.del
-rw-r----- 1 prodedm edm   20 Sep 23 21:27 segments.gen
-rw-r----- 1 prodedm edm 7.7K Sep 23 21:27 segments_18xk


Is there any tool available or any easy way we can remove unnecessary 
files to clean up the index and save some disk space? 

Thanks in advance!
Shlomit
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message