lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Sorting by field is slow
Date Wed, 12 Jun 2013 23:14:14 GMT
This doesn't make much sense, particularly the fact
that you added first/new searchers. I'm assuming that
these are sorting on the same field as your slow query.

But sorting on a text field for which
"Overall, the values of the field are unique"
is a red-flag. Solr doesn't sort on fields that have
more than one term, so you might as well use a
string field and be done with it, it's possible you're
hitting some edge case.

Did you just copy your 3.6 schema and configs to
4.3? Did you re-index?

Best
Erick

On Wed, Jun 12, 2013 at 5:11 PM, Shane Perry <thrykol@gmail.com> wrote:
> Thanks for the responses.
>
> Setting first/newSearcher had no noticeable effect.  I'm sorting on a
> stored/indexed field named 'text' who's fieldType is solr.TextField.
>  Overall, the values of the field are unique. The JVM is only using about
> 2G of the available 12G, so no OOM/GC issue (at least on the surface).  The
> server is question is a slave with approximately 56 million documents.
>  Additionally, sorting on a field of the same type but with significantly
> less uniqueness results quick response times.
>
> The following is a sample of *debugQuery=true* for a query which returns 1
> document:
>
> <lst name="process">
>   <double name="time">61458.0</double>
>   <lst name="query">
>     <double name="time">61452.0</double>
>   </lst>
>   <lst name="facet">
>     <double name="time">0.0</double>
>   </lst>
>   <lst name="mlt">
>     <double name="time">0.0</double>
>   </lst>
>   <lst name="highlight">
>     <double name="time">0.0</double>
>   </lst>
>   <lst name="stats">
>     <double name="time">0.0</double>
>   </lst>
>   <lst name="debug">
>     <double name="time">6.0</double>
>   </lst>
> </lst>
>
>
> -- Update --
>
> Out of desperation, I turned off replication by commenting out the *<list
> name="slave">* element in the replication requestHandler block.  After
> restarting tomcat I was surprised to find that the replication admin UI
> still reported the core as replicating.  Search queries were still slow.  I
> then disabled replication via the UI and the display updated to report the
> core was no longer replicating.  Queries are now fast so it appears that
> the sorting may be a red-herring.
>
> It's may be of note to also mention that the slow queries don't appear to
> be getting cached.
>
> Thanks again for the feed back.
>
> On Wed, Jun 12, 2013 at 2:33 PM, Jack Krupansky <jack@basetechnology.com>wrote:
>
>> Rerun the sorted query with &debugQuery=true and look at the module
>> timings. See what stands out
>>
>> Are you actually sorting on a "text" field, as opposed to a "string" field?
>>
>> Of course, it's always possible that maybe you're hitting some odd OOM/GC
>> condition as a result of Solr growing  between releases.
>>
>> -- Jack Krupansky
>>
>> -----Original Message----- From: Shane Perry
>> Sent: Wednesday, June 12, 2013 3:00 PM
>> To: solr-user@lucene.apache.org
>> Subject: Sorting by field is slow
>>
>>
>> In upgrading from Solr 3.6.1 to 4.3.0, our query response time has
>> increased exponentially.  After testing in 4.3.0 it appears the same query
>> (with 1 matching document) returns after 100 ms without sorting but takes 1
>> minute when sorting by a text field.  I've looked around but haven't yet
>> found a reason for the degradation.  Can someone give me some insight or
>> point me in the right direction for resolving this?  In most cases, I can
>> change my code to do client-side sorting but I do have a couple of
>> situations where pagination prevents client-side sorting.  Any help would
>> be greatly appreciated.
>>
>> Thanks,
>>
>> Shane
>>

Mime
View raw message