Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of vfunstein@gmail.com
 designates 74.125.82.182 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <538C5174.7040101@mailarchiva.com>
References: <01AFE0FB733B9944974A82A09CEB7A0309C81ABB21@mail3.imedx.com>
	<1400570841.2420.155.camel@te-prime>
	<01AFE0FB733B9944974A82A09CEB7A0309C835D126@mail3.imedx.com>
	<1400578244.2420.170.camel@te-prime>
	<01AFE0FB733B9944974A82A09CEB7A0309C835D16A@mail3.imedx.com>
	<1400581087.2420.182.camel@te-prime>
	<01AFE0FB733B9944974A82A09CEB7A0309C835D36D@mail3.imedx.com>
	<1401153243448-4138215.post@n3.nabble.com>
	<003301cf7974$7a200600$6e601200$@gmx.de>
	<01AFE0FB733B9944974A82A09CEB7A0309C881E6CF@mail3.imedx.com>
	<CAN4YXvcH=oKFmgrw2ZiGkiwOc2cq4RR1FQsq4E-Cw-nToA2Sqw@mail.gmail.com>
	<CAN4YXvf8oXwev4U8TErS3mmURwNuja6w2_PB=sxNYoKkqL2_4w@mail.gmail.com>
	<538C1F0F.3010300@mailarchiva.com>
	<CA+iSzr21AX7ZgdMHxGS7=nQqry9SupSqLjN8AFe5CZ1VJR8EfQ@mail.gmail.com>
	<538C5174.7040101@mailarchiva.com>
Date: Tue, 3 Jun 2014 01:54:40 -0700
Message-ID: 
 <CALr4Hzr+qWTi-o2yyBp9Fz_m66oFYXEG=w9G2cT0ecxPUdv3_A@mail.gmail.com>
Subject: Re: search performance
From: Vitaly Funstein <vfunstein@gmail.com>
To: java-user@lucene.apache.org
Content-Type: multipart/alternative; boundary=f46d04428ef684ad8804faeaa96c

--f46d04428ef684ad8804faeaa96c
Content-Type: text/plain; charset=UTF-8

Something doesn't quite add up.

TopFieldCollector fieldCollector = TopFieldCollector.create(sort, max,true,
> false, false, true);
>
> We use pagination, so only returning 1000 documents or so at a time.
>
>
You say you are using pagination, yet the API you are using to create your
collector isn't how you would utilize Lucene's built-in "pagination"
feature (unless misunderstand the API). If the max is the snippet above is
1000, then you're simply returning top 1000 docs every time you execute
your search. Otherwise... well, could you actually post a bit more of your
code that runs the search here, in particular?

Assuming that the max is much larger than 1000, however, you could call
fieldCollector.topDocs(int, int) after accumulating hits using this
collector, but this won't work multiple times per query execution,
according to the javadoc. So you either have to re-execute the full search,
and then get the next chunk of ScoreDocs, or use the proper API for this,
one that accepts as a parameter the end of the previous page of results,
i.e. IndexSearcher.searchAfter(ScoreDoc, ...)

--f46d04428ef684ad8804faeaa96c--