Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of israelekpo@gmail.com designates
 209.85.219.219 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=BUrZNQhq0hl9VA5++j7k7QTAxpzY6fjhdn2RW2Hsvqs4knqnmZfTmm+MxSMvfeinQC
         35MFHBUcOMEarK1moH/2tDQEZv+KJP5IPLnzAeaYQVpyyO6MYC/hQYfVJrflZF59qD+x
         GX8rAVHKoV98c89nvq57idyYDCpMyJsPj67kw=
MIME-Version: 1.0
In-Reply-To: <2613171a0911020652m2f30241cjecef8a77f69a089f@mail.gmail.com>
References: <2613171a0911020513s5f6ddf60i580c5c36e28a073@mail.gmail.com>
	 <c68e39170911020541v75345595l2dc2471839100f38@mail.gmail.com>
	 <d8b8d0160911020545g2f7a4344jbe1b1e76c61ee545@mail.gmail.com>
	 <2613171a0911020652m2f30241cjecef8a77f69a089f@mail.gmail.com>
Date: Mon, 2 Nov 2009 12:16:06 -0500
Message-ID: <d8b8d0160911020916j4b62e35bh29044d4896366e85@mail.gmail.com>
Subject: Re: tracking solr response time
From: Israel Ekpo <israelekpo@gmail.com>
To: solr-user@lucene.apache.org
Content-Type: multipart/alternative; boundary=0016e6d626996f50560477668854

--0016e6d626996f50560477668854
Content-Type: text/plain; charset=UTF-8

On Mon, Nov 2, 2009 at 9:52 AM, bharath venkatesh <
bharathv6.project@gmail.com> wrote:

> Thanks for the quick response
> @yonik
>
> >How much of a latency compared to normal, and what version of Solr are
> you using?
>
> latency is usually around 2-4 secs (some times it goes more than that
> )  which happens  to  only 15-20%  of the request  other  80-85% of
> request are very fast it is in  milli secs ( around 200,000 requests
> happens every day )
>
> @Israel  we are not using java client ..  we  r using  python at the
> client with response formatted in json
>
> @yonikn @Israel   does qtime measure the total time taken at the solr
> server ? I am already measuring the time to get the response  at
> client  end . I would want  a means to know how much time the solr
> server is taking to respond (process ) once it gets the request  . so
> that I could identify whether it is a solr server issue or internal
> network issue
>

It is the time spent at the Solr server.

I think Yonik already answered this part in his response to your thread :

This is what he said :

QTime is the time spent in generating the in-memory representation for
the response before the response writer starts streaming it back in
whatever format was requested.  The stored fields of returned
documents are also loaded at this point (to enable handling of huge
response lists w/o storing all in memory).


>
> @Israel  we are using rhel server  5 on both client and server .. we
> have 6 solr sever . one is acting as master . both client and solr
> sever are on the same network . those servers are dedicated solr
> server except 2 severs which have DB and memcahce running .. we have
> adjusted the load accordingly
>
>
>
>
>
>
>
> On 11/2/09, Israel Ekpo <israelekpo@gmail.com> wrote:
> > On Mon, Nov 2, 2009 at 8:41 AM, Yonik Seeley
> > <yonik@lucidimagination.com>wrote:
> >
> >> On Mon, Nov 2, 2009 at 8:13 AM, bharath venkatesh
> >> <bharathv6.project@gmail.com> wrote:
> >> >    We are using solr for many of ur products  it is doing quite well
> >> > .  But since no of hits are becoming high we are experiencing latency
> >> > in certain requests ,about 15% of our requests are suffering a latency
> >>
> >> How much of a latency compared to normal, and what version of Solr are
> >> you using?
> >>
> >> >  . We are trying to identify  the problem .  It may be due to  network
> >> > issue or solr server is taking time to process the request  .   other
> >> > than  qtime which is returned along with the response is there any
> >> > other way to track solr servers performance ?
> >> > how is qtime calculated
> >> > , is it the total time from when solr server got the request till it
> >> > gave the response ?
> >>
> >> QTime is the time spent in generating the in-memory representation for
> >> the response before the response writer starts streaming it back in
> >> whatever format was requested.  The stored fields of returned
> >> documents are also loaded at this point (to enable handling of huge
> >> response lists w/o storing all in memory).
> >>
> >> There are normally servlet container logs that can be configured to
> >> spit out the real total request time.
> >>
> >> > can we do some extra logging to track solr servers
> >> > performance . ideally I would want to pass some log id along with the
> >> > request (query ) to  solr server  and solr server must log the
> >> > response time along with that log id .
> >>
> >> Yep - Solr isn't bothered by params it doesn't know about, so just put
> >> logid=xxxxxxx and it should also be logged with the other request
> >> params.
> >>
> >> -Yonik
> >> http://www.lucidimagination.com
> >>
> >
> >
> >
> > If you are not using Java then you may have to track the elapsed time
> > manually.
> >
> > If you are using the SolrJ Java client you may have the following
> options:
> >
> > There is a method called getElapsedTime() in
> > org.apache.solr.client.solrj.response.SolrResponseBase which is available
> to
> > all the subclasses
> >
> > I have not used it personally but I think this should return the time
> spent
> > on the client side for that request.
> >
> > The QTime is not the time on the client side but the time spent
> internally
> > at the Solr server to process the request.
> >
> >
> http://lucene.apache.org/solr//api/solrj/org/apache/solr/client/solrj/response/SolrResponseBase.html
> >
> >
> http://lucene.apache.org/solr//api/solrj/org/apache/solr/client/solrj/response/QueryResponse.html
> >
> > Most likely it could be as a result of an internal network issue between
> the
> > two servers or the Solr server is competing with other applications for
> > resources.
> >
> > What operating system is the Solr server running on? Is you client
> > application connection to a Solr server on the same network or over the
> > internet? Are there other applications like database servers etc running
> on
> > the same machine? If so, then the DB server (or any other application)
> and
> > the Solr server could be competing for resources like CPU, memory etc.
> >
> > If you are using Tomcat, you can take a look in
> > $CATALINA_HOME/logs/catalina.out, there are timestamps there that can
> also
> > guide you.
> >
> > --
> > "Good Enough" is not good enough.
> > To give anything less than your best is to sacrifice the gift.
> > Quality First. Measure Twice. Cut Once.
> >
>


-- 
"Good Enough" is not good enough.
To give anything less than your best is to sacrifice the gift.
Quality First. Measure Twice. Cut Once.

--0016e6d626996f50560477668854--