lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: solr 5.2.0 need to build high query response
Date Tue, 05 Jan 2016 18:24:40 GMT
Yep. Do note what's happening here. You're executing a query
that potentially takes 10 seconds to execute (based on your
earlier post). But you may be opening a new searcher every
2 seconds. You may start to see "too many on deck searchers"
in your log. If you do do _not_ try to "fix" this by upping the
maxWarmingSearchers in solrconfig.xml, that's really an
anti-pattern.

Really, I'd consider relaxing this 2 second limit. I've often found
it easier to tell users "it may take up to 30 seconds for newly-added
docs to appear in search results" than try to satisfy overly-tight
requirements.

As a former co-worker often said, "Users are much more comfortable
with predictable delays than unpredictable ones". It's surprising how
often it's the case.

Best,
Erick

P.S. What's the difference between newSearcher and firstSearcher?
newSearcher is fired every time a commit (soft or hard with openSearcher=true)
where firstSearcher is fired up only when Solr starts. This is to accommodate
the fact that the autowarm counts on things like filterCacher aren't
available when Solr starts. In practice, though, many (most?) people
put the same query in both.

On Tue, Jan 5, 2016 at 9:17 AM, Novin Novin <toe.alean@gmail.com> wrote:
> If I'm correct, you are talking about this
>
> <listener event="newSearcher" class="solr.QuerySenderListener">
>             <arr name="queries">
>                 <!--
>
> *And how I can mention my spatial search field here.*
>
>                    <lst><str name="q">solr</str><str name="sort">price
> asc</str></lst>
>                    <lst><str name="q">rocks</str><str name="sort">weight
> asc</str></lst>
>                   -->
>             </arr>
>         </listener>
>         <listener event="firstSearcher" class="solr.QuerySenderListener">
>             <arr name="queries">
>                 <lst>
>
> *or may be here too.*
>
>                     <str name="q">static firstSearcher warming in
> solrconfig.xml</str>
>                 </lst>
>             </arr>
>         </listener>
>
> Thanks,
> Novin
>
> On Tue, 5 Jan 2016 at 16:22 Erick Erickson <erickerickson@gmail.com> wrote:
>
>> It sounds like you're not doing proper autowarming,
>> which you'd need to do either with hard or
>> soft commits that open new searchers.
>>
>> see:
>> https://wiki.apache.org/solr/SolrCaching#Cache_Warming_and_Autowarming
>>
>> In particular, you should have a newSearcher event
>> that facets on the fields you expect to need.
>>
>> Best,
>> Erick
>>
>> On Tue, Jan 5, 2016 at 8:17 AM, Novin Novin <toe.alean@gmail.com> wrote:
>> > Thanks David. It is quite good to use for NRT.
>> >
>> > Apologies, I didn't mention that facet search is really slow.
>> >
>> > I found the below reason which could be the reason because I am using
>> facet
>> > spatial search which is getting slow.
>> >
>> > To know more about solr hard and soft commits, have a look at this blog :
>> >
>> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
>> >
>> > In this article, "soft commits are that they will make documents visible,
>> > but at some cost. In particular the “top level” caches, which include
>> what
>> > you configure in solrconfig.xml (filterCache, queryResultCache, etc) will
>> > be invalidated! Autowarming will be performed on your top level caches
>> > (e.g. filterCache, queryResultCache), and any newSearcher queries will be
>> > executed. Also, the FieldValueCache is invalidated, so facet queries will
>> > have to wait until the cache is refreshed."
>> >
>> > Do you have any idea what could possible be do about this?
>> >
>> >
>> >
>> > On Tue, 5 Jan 2016 at 12:31 davidphilip cherian <
>> > davidphilipcherian@gmail.com> wrote:
>> >
>> >> You should use solr softcommit for this use case. So, by setting
>> softcommit
>> >> to 5 seconds and autoCommit to minute with openSearcher=false should do
>> the
>> >> work.
>> >>
>> >>  <autoCommit>
>> >>  <maxTime>60000</maxTime>
>> >> <openSearcher>false</openSearcher>
>> >>  </autoCommit>
>> >>
>> >> <autoSoftCommit>
>> >> <maxTime>2000</maxTime>
>> >> </autoSoftCommit>
>> >>
>> >> Reference link-
>> >>
>> https://cwiki.apache.org/confluence/display/solr/Near+Real+Time+Searching
>> >>
>> >> To know more about solr hard and soft commits, have a look at this blog
>> :
>> >>
>> >>
>> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
>> >>
>> >> On Tue, Jan 5, 2016 at 5:44 PM, Novin Novin <toe.alean@gmail.com>
>> wrote:
>> >>
>> >> > Hi guys,
>> >> >
>> >> > I'm having trouble to figure what would be idle solr config for where:
>> >> >
>> >> > I'm doing hard commit in every minute   for very few number of users
>> >> > because I have to show those docs in search results quickly when user
>> >> save
>> >> > the changes.
>> >> >
>> >> > It is causing the response in around  2 secs to show even I am getting
>> >> only
>> >> > 10 records.
>> >> >
>> >> > Could you able to give some idea where to look at.
>> >> >
>> >> >
>> >> > Thanks in advance,
>> >> > Novin
>> >> >
>> >>
>>

Mime
View raw message