lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: tracking solr response time
Date Mon, 02 Nov 2009 22:27:47 GMT
So I need someone with better knowledge to chime in here with an opinion
on whether autowarming would help since the whole faceting thing is
something
I'm not very comfortable with...

<hint, hint, hint>

Erick

On Mon, Nov 2, 2009 at 2:21 PM, bharath venkatesh <
bharathv6.project@gmail.com> wrote:

> @Israel: yes I got that point which yonik mentioned .. but is qtime the
> total time taken by solr server for that request or  is it  part of time
> taken by the solr for that request ( is there any thing that a solr server
> does for that particulcar request which is not included in that qtime
> bracket ) ?  I am sorry for dragging in to this qtime. I just want to be
> sure, as we observed many times there is huge mismatch between qtime and
> time measured at the client for the response ( does this imply it is due to
> internal network issue )
>
> @Erick: yes, many times query is slow first time its executed is there any
> solution to improve upon this factor .. for querying we use
> DisMaxRequestHandler , queries are quite long with many faceting parameters
> .
>
>
> On Mon, Nov 2, 2009 at 10:46 PM, Israel Ekpo <israelekpo@gmail.com> wrote:
>
> > On Mon, Nov 2, 2009 at 9:52 AM, bharath venkatesh <
> > bharathv6.project@gmail.com> wrote:
> >
> > > Thanks for the quick response
> > > @yonik
> > >
> > > >How much of a latency compared to normal, and what version of Solr are
> > > you using?
> > >
> > > latency is usually around 2-4 secs (some times it goes more than that
> > > )  which happens  to  only 15-20%  of the request  other  80-85% of
> > > request are very fast it is in  milli secs ( around 200,000 requests
> > > happens every day )
> > >
> > > @Israel  we are not using java client ..  we  r using  python at the
> > > client with response formatted in json
> > >
> > > @yonikn @Israel   does qtime measure the total time taken at the solr
> > > server ? I am already measuring the time to get the response  at
> > > client  end . I would want  a means to know how much time the solr
> > > server is taking to respond (process ) once it gets the request  . so
> > > that I could identify whether it is a solr server issue or internal
> > > network issue
> > >
> >
> > It is the time spent at the Solr server.
> >
> > I think Yonik already answered this part in his response to your thread :
> >
> > This is what he said :
> >
> > QTime is the time spent in generating the in-memory representation for
> > the response before the response writer starts streaming it back in
> > whatever format was requested.  The stored fields of returned
> > documents are also loaded at this point (to enable handling of huge
> > response lists w/o storing all in memory).
> >
> >
> > >
> > > @Israel  we are using rhel server  5 on both client and server .. we
> > > have 6 solr sever . one is acting as master . both client and solr
> > > sever are on the same network . those servers are dedicated solr
> > > server except 2 severs which have DB and memcahce running .. we have
> > > adjusted the load accordingly
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On 11/2/09, Israel Ekpo <israelekpo@gmail.com> wrote:
> > > > On Mon, Nov 2, 2009 at 8:41 AM, Yonik Seeley
> > > > <yonik@lucidimagination.com>wrote:
> > > >
> > > >> On Mon, Nov 2, 2009 at 8:13 AM, bharath venkatesh
> > > >> <bharathv6.project@gmail.com> wrote:
> > > >> >    We are using solr for many of ur products  it is doing quite
> well
> > > >> > .  But since no of hits are becoming high we are experiencing
> > latency
> > > >> > in certain requests ,about 15% of our requests are suffering
a
> > latency
> > > >>
> > > >> How much of a latency compared to normal, and what version of Solr
> are
> > > >> you using?
> > > >>
> > > >> >  . We are trying to identify  the problem .  It may be due to
> >  network
> > > >> > issue or solr server is taking time to process the request  .
> > other
> > > >> > than  qtime which is returned along with the response is there
any
> > > >> > other way to track solr servers performance ?
> > > >> > how is qtime calculated
> > > >> > , is it the total time from when solr server got the request
till
> it
> > > >> > gave the response ?
> > > >>
> > > >> QTime is the time spent in generating the in-memory representation
> for
> > > >> the response before the response writer starts streaming it back in
> > > >> whatever format was requested.  The stored fields of returned
> > > >> documents are also loaded at this point (to enable handling of huge
> > > >> response lists w/o storing all in memory).
> > > >>
> > > >> There are normally servlet container logs that can be configured to
> > > >> spit out the real total request time.
> > > >>
> > > >> > can we do some extra logging to track solr servers
> > > >> > performance . ideally I would want to pass some log id along
with
> > the
> > > >> > request (query ) to  solr server  and solr server must log the
> > > >> > response time along with that log id .
> > > >>
> > > >> Yep - Solr isn't bothered by params it doesn't know about, so just
> put
> > > >> logid=xxxxxxx and it should also be logged with the other request
> > > >> params.
> > > >>
> > > >> -Yonik
> > > >> http://www.lucidimagination.com
> > > >>
> > > >
> > > >
> > > >
> > > > If you are not using Java then you may have to track the elapsed time
> > > > manually.
> > > >
> > > > If you are using the SolrJ Java client you may have the following
> > > options:
> > > >
> > > > There is a method called getElapsedTime() in
> > > > org.apache.solr.client.solrj.response.SolrResponseBase which is
> > available
> > > to
> > > > all the subclasses
> > > >
> > > > I have not used it personally but I think this should return the time
> > > spent
> > > > on the client side for that request.
> > > >
> > > > The QTime is not the time on the client side but the time spent
> > > internally
> > > > at the Solr server to process the request.
> > > >
> > > >
> > >
> >
> http://lucene.apache.org/solr//api/solrj/org/apache/solr/client/solrj/response/SolrResponseBase.html
> > > >
> > > >
> > >
> >
> http://lucene.apache.org/solr//api/solrj/org/apache/solr/client/solrj/response/QueryResponse.html
> > > >
> > > > Most likely it could be as a result of an internal network issue
> > between
> > > the
> > > > two servers or the Solr server is competing with other applications
> for
> > > > resources.
> > > >
> > > > What operating system is the Solr server running on? Is you client
> > > > application connection to a Solr server on the same network or over
> the
> > > > internet? Are there other applications like database servers etc
> > running
> > > on
> > > > the same machine? If so, then the DB server (or any other
> application)
> > > and
> > > > the Solr server could be competing for resources like CPU, memory
> etc.
> > > >
> > > > If you are using Tomcat, you can take a look in
> > > > $CATALINA_HOME/logs/catalina.out, there are timestamps there that can
> > > also
> > > > guide you.
> > > >
> > > > --
> > > > "Good Enough" is not good enough.
> > > > To give anything less than your best is to sacrifice the gift.
> > > > Quality First. Measure Twice. Cut Once.
> > > >
> > >
> >
> >
> >
> > --
> > "Good Enough" is not good enough.
> > To give anything less than your best is to sacrifice the gift.
> > Quality First. Measure Twice. Cut Once.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message