lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Is there a way to limit the size of an index?
Date Thu, 07 Jan 2010 11:32:13 GMT
Is the creation of the index in your control?  If so, try something like this:

  ((LogByteSizeMergePolicy) writer.getMergePolicy()).setMaxMergeMB(1.0));

(NOTE: not tested).  This works because [currently]
LogByteSizeMergePolicy is the default for IndexWriter.  Would be safer
to create your own LogByteSizeMergePolicy and set it on the writer...

This tells the IndexWriter NOT to merge any segment that's over 1.0 MB
in size.  With a default merge factor of 10, this should "generally"
mean you don't get a segment over 10MB, though it may not be a hard
guarantee (you can lower the 1.0 if you still see a segment over 10
MB).

Implementing a custom directory that splits files @ 10 MB is also
possible, but is a good amount more work.  You'd have to have your
IndexOutput write to the separate 10 MB sized files, then make an
IndexInput that "pretends" they are one file.

Mike

On Thu, Jan 7, 2010 at 6:24 AM, Dvora <barak.yaish@gmail.com> wrote:
>
> Thanks for the reply.
>
> Can you please add some detailed explanations? I'm trying to upload Lucene
> index to Google App Engine, and the files size must not exceed 10MB.
>
>
>
> Michael McCandless-2 wrote:
>>
>> I don't think this is implemented [yet] today.  You'd have to
>> implement the Directory, IndexInput and IndexOutput classes, to make
>> this work.
>>
>
> --
> View this message in context: http://old.nabble.com/Is-there-a-way-to-limit-the-size-of-an-index--tp27056573p27058014.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message