Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of yseeley@gmail.com designates
 209.85.219.227 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:reply-to:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=DsFMnh5krzK+AC93a7vvigjp5ORw1iRLfjx/iIn7TFA8mORAygzNLXlSQU2/WeqQTl
         r4ugIgHZSU/qpR3klx0Qtxxr1BPsYcgiyL9x8HC76S1yTc+R4SWZtbk44ciWxhMW/MD2
         Xg8fNKlyTocIrPkQlhV8F9kJERilLJlhTeVLE=
MIME-Version: 1.0
Sender: yseeley@gmail.com
Reply-To: yonik@lucidimagination.com
In-Reply-To: <e994873a0909090640q5206527epeb494ab43c8ebcc0@mail.gmail.com>
References: <4A9829C2.2000104@apache.org>
	 <e994873a0909090557v3a414566k8b24608bfab8b62c@mail.gmail.com>
	 <c68e39170909090617n7dfb031dube6cddb0a6437593@mail.gmail.com>
	 <e994873a0909090640q5206527epeb494ab43c8ebcc0@mail.gmail.com>
Date: Wed, 9 Sep 2009 10:17:21 -0400
Message-ID: <c68e39170909090717w4b9c5423jb45e294c01798032@mail.gmail.com>
Subject: Re: Lucene 2.9 RC2 now available for testing
From: Yonik Seeley <yonik@lucidimagination.com>
To: java-user@lucene.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Wed, Sep 9, 2009 at 9:40 AM, Peter Keegan<peterlkeegan@gmail.com> wrote:
> IndexSearcher.search is calling my custom scorer's 'next' and 'doc' metho=
ds
> 64% fewer times. I see no 'advance' method in any of the hot spots'. I am
> getting the same number of hits from the custom scorer.
> Has the BooleanScorer2 logic changed?

What does your query structure look like?
BS2 hasn't changed much.  The old BS may be used in certain
circumstances, but not in a way that you would see a decrease in
skipTo()+next().

Shot in the dark: if your scorer is part of a conjunction, maybe
you're getting lucky with an optimization:
http://svn.apache.org/viewvc?view=3Drev&revision=3D630698
It was part of http://issues.apache.org/jira/browse/LUCENE-693 , but I
had a slight bug to the "highest skip first" optimization that wasn't
fixed until 2.4

Let's assume that your custom scorer is dense (matches almost
everything) and that another scorer is sparse.  If your custom scorer
appears first, the conjunction scorer logic would be like so:
custom.skipTo(),  scorer1.skipTo(),  custom.skipTo() =3D> match!

If the sparse scorer is first, it will look more like so:
scorer1.skipTo(),  custom.skipTo() =3D> match!

-Yonik
http://www.lucidimagination.com


> Peter
>
> On Wed, Sep 9, 2009 at 9:17 AM, Yonik Seeley <
> yonik.seeley@lucidimagination.com> wrote:
>
>> On Wed, Sep 9, 2009 at 8:57 AM, Peter Keegan<peterlkeegan@gmail.com>
>> wrote:
>> > Using JProfiler, I observe that the improvement
>> > is due to a huge reduction in the number of calls to TermDocs.next and
>> > TermDocs.skipTo (about 65% fewer calls).
>>
>> Indexes are searched per-segment now (i.e. MultiTermDocs isn't normally
>> used).
>> Off the top of my head, I'm not sure how this can lead to fewer
>> TermDocs.skipTo() calls though. =A0Are you sure you weren't also
>> counting Scorer.skipTo()... which would now be Scorer.advance()?
>> Have you verified that your custom scorer is working correctly with
>> 2.9 and that you're getting the same number of hits on the overall
>> query as you were with previous versions?
>>
>> -Yonik
>> http://www.lucidimagination.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org