Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Message-ID;
  b=bNoZLr3xuqoC/RVrFs36bft7FHbisN63l1G7Cou5rwdyxc8mVzNUX8KUHCIBJ9j5VC/1pkaJQAaS+wTbugn1ufl4+RbyUih8KTCFaTRgfx96yDaS0vyoIfjRcu/RP7wcwINHVJz7yWuacP4ba6cysKZ+Lwx302E1hxiyOZn+YwU=;
Date: Mon, 24 Mar 2008 18:34:48 -0700 (PDT)
From: Otis Gospodnetic <otis_gospodnetic@yahoo.com>
Subject: Re: how to control the disk size of the indices
To: general@lucene.apache.org
Cc: java-user@lucene.apache.org
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Message-ID: <331829.59350.qm@web50304.mail.re2.yahoo.com>

Hi Yannis,

I don't think there is anything of that sort in Lucene, but this shouldn't be hard to do with a process outside Lucene.  Of course. optimizing an index increases its size temporarily, so your external process would have to take that into account and play it safe.  You could also set mergeFactor to 1, which should keep your index in a fully optimized state if you don't do any deletions and near-optimized state if you do deletions.

You should discuss this on java-user list, though, so I'm CCing that list where you can continue the discussion.

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch

----- Original Message ----
From: Yannis Pavlidis <ypavlidis@me.dium.com>
To: general@lucene.apache.org
Sent: Monday, March 24, 2008 7:33:26 PM
Subject: how to control the disk size of the indices


Hi all,

I wanted to ask the list whether there is an easy and efficient way to manage the size (in bytes) of a lucene index stored on disk.

Basically I would like to limit lucene storing only 100 GB of information. When lucene reaches that limit then I would delete the documents (using an LRU algorithm based on timestaps) but in no case the disk space occupied by Lucene should exceed 100GB.

I experimented with lucene 2.3.1 and the only I could accomplish that was by calling the optimize method (after the index size exceeded the max size) on the IndexWriter. I was looking for a more performant way to "perhaps control" Lucene on when to merge the segments so as to not exceed the pre-set limit.

Any ideas or suggestions would be highly appreciated.

Thanks in advance,

Yannis.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org