lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <>
Subject Re: lucene 2.9 sorting algorithm
Date Fri, 23 Oct 2009 12:37:34 GMT
Yup, I'm not against the testing or the thought - and it is clearly more
complicated - I'm not saying its not. But I haven't seen anyone thats
come and said they haven't grokked it yet or that they had a hard time
with it (though they have run into limitations in what they have tried
to do). John and Jake are not the first the upgrade (and while they have
noted its more complicated, they grokked it too).

Its a matter of how difficult it is, not thats its a little more
complicated than the old. Doing this is an advanced Lucene op - a bunch
of people have done it already - these guys are not the first - no one
has really gotten tripped up.

I'm not in love with the new API, but I'm still waiting to see the list
of people that haven't grokked it.

I don't like the two idea of supporting two API's and I don't like the
idea of herding everyone back right after herding them over. If it
clearly made sense, I have no reason to be against it - but I'm just not
seeing that myself.

Michael McCandless wrote:
> Sheesh I go to bed and so much all of a sudden happens!!
> Sorry Mark; I should've called out "PATCH IS ON 2.9 BRANCH" more
> clearly ;)
> There's no question in my mind that the new comparator API is more
> complex than the old one, and I really don't like that.  I had to
> rewrite the section of LIA that gives an example of a [simple] custom
> sort and it wasn't pleasant!  Two compare methods (compare,
> compareBottom)?  Two copy methods (copy, setBottom)?  Sure, you can
> grok it and get through it if you have to, but it is more complex
> because it's conflated with the PQ API.
> Ease on consumption of our APIs is very important, so, only when
> performance clearly warrants it should we adopt a more complex API.
> Also, yeah, it would suck to have to switch back to the old API at
> this point, but net/net I still think we should if performance is no
> better with the new one.
> The old API also fits cleanly with per-segment searching (John's
> initial patch shows that -- it's simply another per-segment Colletor).
> The two APIs (collection, comparator) are well decoupled.
> So, digging in deep and running thorough perf tests makes sense; we
> need to understand the performance to make the API switch decision.
> And definitely we should tune both approaches as much as possible
> (removing that if from the Multi PQ patch makes sense).
> But... Multi PQ's performance isn't better in many cases... though,
> we're clearly still iterating.  I'll run a 1.5 (32 & 64 bit) test,
> with the if statement removed.
> Mike
> On Fri, Oct 23, 2009 at 3:53 AM, Earwin Burrfoot <> wrote:
>> I did.
>> On Fri, Oct 23, 2009 at 09:05, Jake Mannix <> wrote:
>>> On Thu, Oct 22, 2009 at 9:58 PM, Mark Miller <> wrote:
>>>> Yes - I've seen a handful of non core devs report back that they
>>>> upgraded with no complaints on the difficulty. Its in the mailing list
>>>> archives. The only core dev I've seen say its easy is Uwe. He's super
>>>> sharp though, so I wasn't banking my comment on him ;)
>>> Upgrade custom sorting?  Where has anyone talked about this?
>>> 2.9 is great, I like the new apis, they're great in general.  It's just this
>>> multi-segment sorting we're talking about here.
>>>   -jake
>> --
>> Kirill Zakharenko/Кирилл Захаренко (
>> Home / Mobile: +7 (495) 683-567-4 / +7 (903) 5-888-423
>> ICQ: 104465785
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

- Mark

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message