lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From deminix <demi...@gmail.com>
Subject Re: Term Limit?
Date Sat, 04 Apr 2009 16:29:25 GMT
Ah yes.  I'd be happy with the ability to monitor it for now.  Assuming it
is too involved to remove the limitation.

For all practical purposes we should only be using, worst case, 10% of the
term space today.  That happens to make it risky enough that it needs an eye
kept on it, as this will be our authoritative store and the data isn't
within our control.

I'll see what I can do about trying to break it during perf testing ;)


On Sat, Apr 4, 2009 at 9:06 AM, Michael McCandless <
lucene@mikemccandless.com> wrote:

> On Sat, Apr 4, 2009 at 11:57 AM, deminix <deminix@gmail.com> wrote:
> > Yea.  That is all that matters anyway right, is the limit at the segment
> > level?
>
> Well... the problem is when merges kick off.
>
> You could have N segments that each are below the limit, but when a
> merge runs the merged segment would try to exceed the limit, because
> you can't easily predict how many unique terms the merged segment will
> have.
>
> I guess you could guarantee safety by never allowing the sum of the
> unique term count across all segments to exceed 2^31-1, but that may
> be a too-restrictive safety margin.
>
> Mike
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message