lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Novin Novin <toe.al...@gmail.com>
Subject Re: solr 5.2.0 need to build high query response
Date Wed, 06 Jan 2016 10:15:11 GMT
Thanks Erick, this listener doing quite a good job. But not what  I needed.
Do the solr has any other things that I can look into to make it faster.
FYI  speed goes to 1 sec to 1.2 sec. I actually needed around 500 ms.

On Tue, 5 Jan 2016 at 18:24 Erick Erickson <erickerickson@gmail.com> wrote:

> Yep. Do note what's happening here. You're executing a query
> that potentially takes 10 seconds to execute (based on your
> earlier post). But you may be opening a new searcher every
> 2 seconds. You may start to see "too many on deck searchers"
> in your log. If you do do _not_ try to "fix" this by upping the
> maxWarmingSearchers in solrconfig.xml, that's really an
> anti-pattern.
>
> Really, I'd consider relaxing this 2 second limit. I've often found
> it easier to tell users "it may take up to 30 seconds for newly-added
> docs to appear in search results" than try to satisfy overly-tight
> requirements.
>
> As a former co-worker often said, "Users are much more comfortable
> with predictable delays than unpredictable ones". It's surprising how
> often it's the case.
>
> Best,
> Erick
>
> P.S. What's the difference between newSearcher and firstSearcher?
> newSearcher is fired every time a commit (soft or hard with
> openSearcher=true)
> where firstSearcher is fired up only when Solr starts. This is to
> accommodate
> the fact that the autowarm counts on things like filterCacher aren't
> available when Solr starts. In practice, though, many (most?) people
> put the same query in both.
>
> On Tue, Jan 5, 2016 at 9:17 AM, Novin Novin <toe.alean@gmail.com> wrote:
> > If I'm correct, you are talking about this
> >
> > <listener event="newSearcher" class="solr.QuerySenderListener">
> >             <arr name="queries">
> >                 <!--
> >
> > *And how I can mention my spatial search field here.*
> >
> >                    <lst><str name="q">solr</str><str name="sort">price
> > asc</str></lst>
> >                    <lst><str name="q">rocks</str><str name="sort">weight
> > asc</str></lst>
> >                   -->
> >             </arr>
> >         </listener>
> >         <listener event="firstSearcher" class="solr.QuerySenderListener">
> >             <arr name="queries">
> >                 <lst>
> >
> > *or may be here too.*
> >
> >                     <str name="q">static firstSearcher warming in
> > solrconfig.xml</str>
> >                 </lst>
> >             </arr>
> >         </listener>
> >
> > Thanks,
> > Novin
> >
> > On Tue, 5 Jan 2016 at 16:22 Erick Erickson <erickerickson@gmail.com>
> wrote:
> >
> >> It sounds like you're not doing proper autowarming,
> >> which you'd need to do either with hard or
> >> soft commits that open new searchers.
> >>
> >> see:
> >> https://wiki.apache.org/solr/SolrCaching#Cache_Warming_and_Autowarming
> >>
> >> In particular, you should have a newSearcher event
> >> that facets on the fields you expect to need.
> >>
> >> Best,
> >> Erick
> >>
> >> On Tue, Jan 5, 2016 at 8:17 AM, Novin Novin <toe.alean@gmail.com>
> wrote:
> >> > Thanks David. It is quite good to use for NRT.
> >> >
> >> > Apologies, I didn't mention that facet search is really slow.
> >> >
> >> > I found the below reason which could be the reason because I am using
> >> facet
> >> > spatial search which is getting slow.
> >> >
> >> > To know more about solr hard and soft commits, have a look at this
> blog :
> >> >
> >>
> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
> >> >
> >> > In this article, "soft commits are that they will make documents
> visible,
> >> > but at some cost. In particular the “top level” caches, which include
> >> what
> >> > you configure in solrconfig.xml (filterCache, queryResultCache, etc)
> will
> >> > be invalidated! Autowarming will be performed on your top level caches
> >> > (e.g. filterCache, queryResultCache), and any newSearcher queries
> will be
> >> > executed. Also, the FieldValueCache is invalidated, so facet queries
> will
> >> > have to wait until the cache is refreshed."
> >> >
> >> > Do you have any idea what could possible be do about this?
> >> >
> >> >
> >> >
> >> > On Tue, 5 Jan 2016 at 12:31 davidphilip cherian <
> >> > davidphilipcherian@gmail.com> wrote:
> >> >
> >> >> You should use solr softcommit for this use case. So, by setting
> >> softcommit
> >> >> to 5 seconds and autoCommit to minute with openSearcher=false should
> do
> >> the
> >> >> work.
> >> >>
> >> >>  <autoCommit>
> >> >>  <maxTime>60000</maxTime>
> >> >> <openSearcher>false</openSearcher>
> >> >>  </autoCommit>
> >> >>
> >> >> <autoSoftCommit>
> >> >> <maxTime>2000</maxTime>
> >> >> </autoSoftCommit>
> >> >>
> >> >> Reference link-
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/solr/Near+Real+Time+Searching
> >> >>
> >> >> To know more about solr hard and soft commits, have a look at this
> blog
> >> :
> >> >>
> >> >>
> >>
> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
> >> >>
> >> >> On Tue, Jan 5, 2016 at 5:44 PM, Novin Novin <toe.alean@gmail.com>
> >> wrote:
> >> >>
> >> >> > Hi guys,
> >> >> >
> >> >> > I'm having trouble to figure what would be idle solr config for
> where:
> >> >> >
> >> >> > I'm doing hard commit in every minute   for very few number of
> users
> >> >> > because I have to show those docs in search results quickly when
> user
> >> >> save
> >> >> > the changes.
> >> >> >
> >> >> > It is causing the response in around  2 secs to show even I am
> getting
> >> >> only
> >> >> > 10 records.
> >> >> >
> >> >> > Could you able to give some idea where to look at.
> >> >> >
> >> >> >
> >> >> > Thanks in advance,
> >> >> > Novin
> >> >> >
> >> >>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message