Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
MIME-Version: 1.0
In-Reply-To: <CAN+whnJLRiTzmzDt3r=OjRN9bxBtC6KdRO35HJB_CHBzRXdG3A@mail.gmail.com>
References: <CAN+whnKB+ckg_FKR1mVFvh4js3-BBW2z4V2tnfhe5gSBY6bgAw@mail.gmail.com>
 <CAN4YXvc98yBTiLg=oJFfkdZ13CrsjtPDK-xeX3GNgZ3RCyajzw@mail.gmail.com>
 <CAN+whn+azimsrCgam6ke_-UfzS2dq1VycfkbSNig5pRSd3Fn0A@mail.gmail.com> <CAN+whnJLRiTzmzDt3r=OjRN9bxBtC6KdRO35HJB_CHBzRXdG3A@mail.gmail.com>
From: Nawab Zada Asad Iqbal <khichi@gmail.com>
Date: Fri, 18 Aug 2017 11:07:04 -0700
Message-ID: <CAN+whnJ4bYJJP_X0-USnkV6XHpcZbJf=C4zFCAZQHADvX=8EYw@mail.gmail.com>
Subject: Re: Request Highlighting only for the final set of rows
To: solr-user@lucene.apache.org
Content-Type: multipart/alternative; boundary="001a11c1106a18b04d05570aff45"
archived-at: Fri, 18 Aug 2017 18:07:18 -0000

--001a11c1106a18b04d05570aff45
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Actually, part of me is thinking that there are valid use cases for having
fl and hl.fl with different values. e.g, receive name etc. in =E2=80=9Cclea=
n=E2=80=9D form
in fl field and receive both name and address in html formatted form (by
specifying in hl.fl)


On Fri, Aug 18, 2017 at 10:57 AM, Nawab Zada Asad Iqbal <khichi@gmail.com>
wrote:

> Actually, i realize that it is an incorrect use on my part to pass only
> id+score in fl and specify more fields in the hl.fl fields. This was
> somehow supported in older versions but the new behavior is actually a
> performance improvement for the scenario when user is asking for only ids=
.
>
>
> Nawab
>
> On Fri, Aug 18, 2017 at 8:33 AM, Nawab Zada Asad Iqbal <khichi@gmail.com>
> wrote:
>
>> Thanks Erick for the pointing to better option. I will explore that.
>> After your email, I found that if i have specified 'fl=3D*' in the query=
 then
>> it is doing the right thing (a 2 pass process). However, my queries had
>> 'fl=3Did+score' (or sometimes fl=3Did&fl=3Dscore), in both of these case=
s I found
>> that the shards are asked for highlighting all the results on the first
>> request (and there is no second request).
>>
>> The fl=3D* query is (in my sample case) finishing in 100 msec while same
>> query with fl=3Did+score finishes in 1200 msec.
>>
>> Here are the two queries;
>>
>> http://solrdev.test.net:8984/solr/filesearch/select?&hl=3Don&f
>> l=3D*&start=3D200&rows=3D200&q=3Dnawab&shards=3Dsolrdev.test.net:8984/
>> solr/filesearch,solrdev.test.net:8985/solr/filesearch,solrd
>> ev.test.net:8986/solr/filesearch&wt=3Djson
>>
>>
>> http://solrdev.test.net:8984/solr/filesearch/select?&hl=3Don&f
>> l=3Did&fl=3Dscore&start=3D200&rows=3D200&q=3Dnawab&shards=3Dsolrdev.test
>> .net:8984/solr/filesearch,solrdev.test.net:8985/solr/filesea
>> rch,solrdev.test.net:8986/solr/filesearch&wt=3Djson
>>
>>
>> Thanks
>> Nawab
>>
>>
>>
>>
>> On Fri, Aug 18, 2017 at 7:23 AM, Erick Erickson <erickerickson@gmail.com=
>
>> wrote:
>>
>>> I don't think you're reading it correctly. First of all, if you're
>>> going to do be doing deep paging you should be using cusorMark, see:
>>> https://cwiki.apache.org/confluence/display/solr/Pagination+of+Results.
>>>
>>> Second, it's a two-pass process if you don't use cursormark. The first
>>> pass gets the candidate docs from each shard. But all it returns is
>>> the ID and sort criteria. Then the aggregator node gets the _true_ top
>>> N after sorting all the lists from each shard and issues a second
>>> request for _only_ those docs that have made the top N from each sub
>>> shard, and those should be the only ones highlighted.
>>>
>>> Do you have any evidence to the contrary that they're all being
>>> highlighted? Or are you misinterpreting the log message for the first
>>> pass?
>>>
>>> Best,
>>> Erick
>>>
>>> On Thu, Aug 17, 2017 at 5:43 PM, Nawab Zada Asad Iqbal <khichi@gmail.co=
m>
>>> wrote:
>>> > Hi,
>>> >
>>> > In a multi-node solr installation (without SolrCloud), during a pagin=
g
>>> > scenario (e.g., start=3D1000, rows=3D200), the primary node asks for =
1200
>>> rows
>>> > from each shard. If highlighting is ON, then the primary node is
>>> asking for
>>> > highlighting all the 1200 results from each shard, which doesn't scal=
e
>>> > well. Is there a way to break the shard query in two steps e.g. ask
>>> for the
>>> > 1200 rows and after sorting the 1200 responses from each shard and
>>> finding
>>> > final rows to return (1001 to 1200) , issue another query to shards f=
or
>>> > asking highlighted response for the relevant docs?
>>> >
>>> >
>>> >
>>> > Thanks
>>> > Nawab
>>>
>>
>>
>

--001a11c1106a18b04d05570aff45--