lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <>
Subject Re: lucene 2.9 sorting algorithm
Date Tue, 20 Oct 2009 12:21:41 GMT
Ahhh - I see - way at the top. Man that was early. Had forgotten about
that stuff even before the issue was finished.

Mark Miller wrote:
> Hmm - perhaps I'm not remembering right. Or perhaps we had different
> motivations ;) I never did anything in 1483 based on search perf - and I
> took your tests as testing that we didn't lose perf, not that we gained
> any. The fact that there were some wins was just a nice surprise from my
> perspective.
> A quote from you in that issue:
> "I didn't expect such performance gain (I was hoping for not much
> performance loss, actually). I think it may be that although the
> initial value copy adds some cost, the within-queue comparsions are
> then faster because you don't have to deref back to the fieldcache
> array. It seems we keep accidentally discovering performance gains
> here"
> My whole memory of that issue is that we didn't do anything for
> performance gains. We just happened to measure a few. It was just to get
> to per segment. Was a long time ago though.
> Michael McCandless wrote:
>> On Tue, Oct 20, 2009 at 6:51 AM, Mark Miller <> wrote:
>>> I didn't really follow that thread either - but we didn't move to the new
>>> Comp Api because of it's perfomance vs the old.
>> We did (LUCENE-1483), but those perf tests mixed in a number of other
>> improvements (eg, searching by segment avoids the 2nd pass of
>>[], int[]), whereas John's tests more
>> specifically test the perf difference between single-PQ vs
>> multi-PQ-then-merge (much simpler comparator API).
>> So we are re-scrutinizing that difference... and if the perf gains are
>> minimal or non-existent I think we should still consider going back to
>> the simpler API.
>> I'm working now to set up a full benchmark across real (wikipedia) /
>> synthetic source, different queries, different sorts, balanced vs
>> unbalanced segment sizes, etc.
>> Mike
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:

- Mark

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message