lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <markrmil...@gmail.com>
Subject Re: lucene 2.9 sorting algorithm
Date Tue, 20 Oct 2009 12:21:41 GMT
Ahhh - I see - way at the top. Man that was early. Had forgotten about
that stuff even before the issue was finished.

Mark Miller wrote:
> Hmm - perhaps I'm not remembering right. Or perhaps we had different
> motivations ;) I never did anything in 1483 based on search perf - and I
> took your tests as testing that we didn't lose perf, not that we gained
> any. The fact that there were some wins was just a nice surprise from my
> perspective.
>
> A quote from you in that issue:
>
> "I didn't expect such performance gain (I was hoping for not much
> performance loss, actually). I think it may be that although the
> initial value copy adds some cost, the within-queue comparsions are
> then faster because you don't have to deref back to the fieldcache
> array. It seems we keep accidentally discovering performance gains
> here"
>
> My whole memory of that issue is that we didn't do anything for
> performance gains. We just happened to measure a few. It was just to get
> to per segment. Was a long time ago though.
>
> Michael McCandless wrote:
>   
>> On Tue, Oct 20, 2009 at 6:51 AM, Mark Miller <markrmiller@gmail.com> wrote:
>>   
>>     
>>> I didn't really follow that thread either - but we didn't move to the new
>>> Comp Api because of it's perfomance vs the old.
>>>     
>>>       
>> We did (LUCENE-1483), but those perf tests mixed in a number of other
>> improvements (eg, searching by segment avoids the 2nd pass of
>> MultiTermDocs.read(int[], int[]), whereas John's tests more
>> specifically test the perf difference between single-PQ vs
>> multi-PQ-then-merge (much simpler comparator API).
>>
>> So we are re-scrutinizing that difference... and if the perf gains are
>> minimal or non-existent I think we should still consider going back to
>> the simpler API.
>>
>> I'm working now to set up a full benchmark across real (wikipedia) /
>> synthetic source, different queries, different sorts, balanced vs
>> unbalanced segment sizes, etc.
>>
>> Mike
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>   
>>     
>
>
>   


-- 
- Mark

http://www.lucidimagination.com




---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message