Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
MIME-Version: 1.0
References: 
 <CAO0nwggNBoA3Wyt+nuZycfcsqKE7Br2d=WoT-oSaLe7v4r4rig@mail.gmail.com>
 <CAA0RL5oX_8jVxLhXANX4DjqjQFu-vaJssmc4Hbdz7NWq8n0big@mail.gmail.com>
 <CAO0nwgiHPV=4RkLq6E5E+uOLjZeMdx3gfjpbX8CX+zE_YFWUsQ@mail.gmail.com>
 <CAN4YXvf44A0aD2AhhF3yNaqBcnFTh_S=FW30-whf0W6jUaLqWQ@mail.gmail.com>
 <CAO0nwggwmkE0NrGuFLxTiWC23J8acp5A=+rqFT+0Se+dfpCeKQ@mail.gmail.com>
 <CAN4YXveE5oO-ZzvV8F7ZKH_tFM-6t0sdPw8vyn=rgjYp-f_Z6w@mail.gmail.com>
In-Reply-To: 
 <CAN4YXveE5oO-ZzvV8F7ZKH_tFM-6t0sdPw8vyn=rgjYp-f_Z6w@mail.gmail.com>
From: Novin Novin <toe.alean@gmail.com>
Date: Wed, 06 Jan 2016 10:15:11 +0000
Message-ID: 
 <CAO0nwghtODQyjsOBo=q_FYU+n2sS21TfOvbiDjcUrMrGW-j0Bw@mail.gmail.com>
Subject: Re: solr 5.2.0 need to build high query response
To: solr-user <solr-user@lucene.apache.org>
Content-Type: multipart/alternative; boundary=001a113db368b0806d0528a7a150

--001a113db368b0806d0528a7a150
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Thanks Erick, this listener doing quite a good job. But not what  I needed.
Do the solr has any other things that I can look into to make it faster.
FYI  speed goes to 1 sec to 1.2 sec. I actually needed around 500 ms.

On Tue, 5 Jan 2016 at 18:24 Erick Erickson <erickerickson@gmail.com> wrote:

> Yep. Do note what's happening here. You're executing a query
> that potentially takes 10 seconds to execute (based on your
> earlier post). But you may be opening a new searcher every
> 2 seconds. You may start to see "too many on deck searchers"
> in your log. If you do do _not_ try to "fix" this by upping the
> maxWarmingSearchers in solrconfig.xml, that's really an
> anti-pattern.
>
> Really, I'd consider relaxing this 2 second limit. I've often found
> it easier to tell users "it may take up to 30 seconds for newly-added
> docs to appear in search results" than try to satisfy overly-tight
> requirements.
>
> As a former co-worker often said, "Users are much more comfortable
> with predictable delays than unpredictable ones". It's surprising how
> often it's the case.
>
> Best,
> Erick
>
> P.S. What's the difference between newSearcher and firstSearcher?
> newSearcher is fired every time a commit (soft or hard with
> openSearcher=3Dtrue)
> where firstSearcher is fired up only when Solr starts. This is to
> accommodate
> the fact that the autowarm counts on things like filterCacher aren't
> available when Solr starts. In practice, though, many (most?) people
> put the same query in both.
>
> On Tue, Jan 5, 2016 at 9:17 AM, Novin Novin <toe.alean@gmail.com> wrote:
> > If I'm correct, you are talking about this
> >
> > <listener event=3D"newSearcher" class=3D"solr.QuerySenderListener">
> >             <arr name=3D"queries">
> >                 <!--
> >
> > *And how I can mention my spatial search field here.*
> >
> >                    <lst><str name=3D"q">solr</str><str name=3D"sort">pr=
ice
> > asc</str></lst>
> >                    <lst><str name=3D"q">rocks</str><str name=3D"sort">w=
eight
> > asc</str></lst>
> >                   -->
> >             </arr>
> >         </listener>
> >         <listener event=3D"firstSearcher" class=3D"solr.QuerySenderList=
ener">
> >             <arr name=3D"queries">
> >                 <lst>
> >
> > *or may be here too.*
> >
> >                     <str name=3D"q">static firstSearcher warming in
> > solrconfig.xml</str>
> >                 </lst>
> >             </arr>
> >         </listener>
> >
> > Thanks,
> > Novin
> >
> > On Tue, 5 Jan 2016 at 16:22 Erick Erickson <erickerickson@gmail.com>
> wrote:
> >
> >> It sounds like you're not doing proper autowarming,
> >> which you'd need to do either with hard or
> >> soft commits that open new searchers.
> >>
> >> see:
> >> https://wiki.apache.org/solr/SolrCaching#Cache_Warming_and_Autowarming
> >>
> >> In particular, you should have a newSearcher event
> >> that facets on the fields you expect to need.
> >>
> >> Best,
> >> Erick
> >>
> >> On Tue, Jan 5, 2016 at 8:17 AM, Novin Novin <toe.alean@gmail.com>
> wrote:
> >> > Thanks David. It is quite good to use for NRT.
> >> >
> >> > Apologies, I didn't mention that facet search is really slow.
> >> >
> >> > I found the below reason which could be the reason because I am usin=
g
> >> facet
> >> > spatial search which is getting slow.
> >> >
> >> > To know more about solr hard and soft commits, have a look at this
> blog :
> >> >
> >>
> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-sof=
tcommit-and-commit-in-sorlcloud/
> >> >
> >> > In this article, "soft commits are that they will make documents
> visible,
> >> > but at some cost. In particular the =E2=80=9Ctop level=E2=80=9D cach=
es, which include
> >> what
> >> > you configure in solrconfig.xml (filterCache, queryResultCache, etc)
> will
> >> > be invalidated! Autowarming will be performed on your top level cach=
es
> >> > (e.g. filterCache, queryResultCache), and any newSearcher queries
> will be
> >> > executed. Also, the FieldValueCache is invalidated, so facet queries
> will
> >> > have to wait until the cache is refreshed."
> >> >
> >> > Do you have any idea what could possible be do about this?
> >> >
> >> >
> >> >
> >> > On Tue, 5 Jan 2016 at 12:31 davidphilip cherian <
> >> > davidphilipcherian@gmail.com> wrote:
> >> >
> >> >> You should use solr softcommit for this use case. So, by setting
> >> softcommit
> >> >> to 5 seconds and autoCommit to minute with openSearcher=3Dfalse sho=
uld
> do
> >> the
> >> >> work.
> >> >>
> >> >>  <autoCommit>
> >> >>  <maxTime>60000</maxTime>
> >> >> <openSearcher>false</openSearcher>
> >> >>  </autoCommit>
> >> >>
> >> >> <autoSoftCommit>
> >> >> <maxTime>2000</maxTime>
> >> >> </autoSoftCommit>
> >> >>
> >> >> Reference link-
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/solr/Near+Real+Time+Searching
> >> >>
> >> >> To know more about solr hard and soft commits, have a look at this
> blog
> >> :
> >> >>
> >> >>
> >>
> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-sof=
tcommit-and-commit-in-sorlcloud/
> >> >>
> >> >> On Tue, Jan 5, 2016 at 5:44 PM, Novin Novin <toe.alean@gmail.com>
> >> wrote:
> >> >>
> >> >> > Hi guys,
> >> >> >
> >> >> > I'm having trouble to figure what would be idle solr config for
> where:
> >> >> >
> >> >> > I'm doing hard commit in every minute   for very few number of
> users
> >> >> > because I have to show those docs in search results quickly when
> user
> >> >> save
> >> >> > the changes.
> >> >> >
> >> >> > It is causing the response in around  2 secs to show even I am
> getting
> >> >> only
> >> >> > 10 records.
> >> >> >
> >> >> > Could you able to give some idea where to look at.
> >> >> >
> >> >> >
> >> >> > Thanks in advance,
> >> >> > Novin
> >> >> >
> >> >>
> >>
>

--001a113db368b0806d0528a7a150--