lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shai Erera <ser...@gmail.com>
Subject Re: 140GB index directory, what can I do?
Date Sat, 14 Aug 2010 11:01:30 GMT
You can also call deleteUnusedFiles(), and all unreferenced files will be
deleted either. Make sure to set the index DeletionPolicy to
KeepOnlyLastCommit (which is the default), before you do that. That's
relevant though if you've built the index using either 3x or 4.0 code.

If not, you can achieve the same goal by, say, add a document, commit,
delete the document and commit again. The first commit will delete the
unused files. Perhaps if you simply do new IndexWriter().close() it will
delete the unused files as well, but I'm not sure.

Shai

On Sat, Aug 14, 2010 at 1:15 PM, Uwe Schindler <uwe@thetaphi.de> wrote:

> Optimize the index one time then all unused segment files are *for-sure*
> removed.
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
> > -----Original Message-----
> > From: Andrew Bruno [mailto:andrew.bruno@gmail.com]
> > Sent: Saturday, August 14, 2010 12:11 PM
> > To: java-user@lucene.apache.org
> > Subject: 140GB index directory, what can I do?
> >
> > Hi all,
> >
> > I have an index directory that is growing pretty fast, and is now at
> 138GB.
> >
> > A while ago, this index got corrupted.  It was rebuilt, but the engineer
> cannot
> > remember whether he deleted the corrupt directory before the rebuild.  Is
> > there a way to know if any files are not being used or whether they can
> be
> > purged, merged, etc?
> >
> > The files are listed below.
> >
> > 08/14/2010  02:51 AM    <DIR>          ..
> > 03/24/2010  11:22 PM                 0 clearcache
> > 08/14/2010  02:51 AM                 4 deletable
> > 08/14/2010  02:51 AM               369 segments
> > 03/29/2010  02:29 AM   135,941,156,712 _1x3.cfs
> > 08/13/2010  08:24 AM           275,615 _1x3.del
> > 05/10/2010  09:59 AM       977,999,933 _491b.cfs
> > 08/10/2010  12:46 PM            17,304 _491b.del
> > 06/18/2010  09:33 PM       809,473,470 _7sr7.cfs
> > 08/06/2010  07:10 AM            15,038 _7sr7.del
> > 07/23/2010  11:34 PM       559,421,479 _aeps.cfs
> > 08/13/2010  07:20 AM            11,276 _aeps.del
> > 08/09/2010  09:23 AM       299,717,058 _bu0l.cfs
> > 08/12/2010  12:42 PM             6,387 _bu0l.del
> > 08/11/2010  02:46 PM        66,910,259 _c4x4.cfs
> > 08/11/2010  04:06 PM             1,401 _c4x4.del
> > 08/13/2010  02:11 PM        45,009,744 _cc9c.cfs
> > 08/13/2010  03:59 PM               944 _cc9c.del
> > 08/13/2010  04:03 PM         3,409,701 _ccqa.cfs
> > 08/13/2010  04:29 PM                79 _ccqa.del
> > 08/13/2010  10:42 PM           900,521 _ccvl.cfs
> > 08/14/2010  12:01 AM                29 _ccvl.del
> > 08/14/2010  12:00 AM           589,015 _cczi.cfs
> > 08/14/2010  12:07 AM                24 _cczi.del
> > 08/14/2010  12:05 AM           260,231 _cd0v.cfs
> > 08/14/2010  02:44 AM           324,498 _cd1z.cfs
> > 08/14/2010  02:44 AM            10,651 _cd20.cfs
> > 08/14/2010  02:45 AM             6,427 _cd21.cfs
> > 08/14/2010  02:45 AM             6,702 _cd22.cfs
> > 08/14/2010  02:45 AM             9,024 _cd23.cfs
> > 08/14/2010  02:45 AM            10,641 _cd24.cfs
> > 08/14/2010  02:46 AM             9,037 _cd25.cfs
> > 08/14/2010  02:46 AM             6,297 _cd26.cfs
> > 08/14/2010  02:46 AM             6,504 _cd27.cfs
> > 08/14/2010  02:46 AM            56,164 _cd28.cfs
> > 08/14/2010  02:47 AM                 9 _cd28.del
> > 08/14/2010  02:47 AM            56,246 _cd29.cfs
> > 08/14/2010  02:47 AM             7,093 _cd2a.cfs
> > 08/14/2010  02:47 AM                 9 _cd2a.del
> > 08/14/2010  02:47 AM             7,179 _cd2b.cfs
> > 08/14/2010  02:48 AM            10,324 _cd2c.cfs
> > 08/14/2010  02:48 AM                 9 _cd2c.del
> > 08/14/2010  02:48 AM            10,412 _cd2d.cfs
> > 08/14/2010  02:49 AM            67,808 _cd2e.cfs
> > 08/14/2010  02:49 AM             6,431 _cd2f.cfs
> > 08/14/2010  02:49 AM             6,419 _cd2g.cfs
> > 08/14/2010  02:49 AM            10,394 _cd2h.cfs
> > 08/14/2010  02:49 AM            10,417 _cd2i.cfs
> > 08/14/2010  02:49 AM             8,379 _cd2j.cfs
> > 08/14/2010  02:50 AM             5,982 _cd2k.cfs
> > 08/14/2010  02:50 AM             5,670 _cd2l.cfs
> > 08/14/2010  02:51 AM             5,622 _cd2m.cfs
> >               51 File(s) 138,705,840,941 bytes
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message