Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: local policy)
From: Toke Eskildsen <te@statsbiblioteket.dk>
To: "solr-user@lucene.apache.org" <solr-user@lucene.apache.org>
Date: Tue, 24 Feb 2015 21:34:24 +0100
Subject: RE: how to debug solr performance degradation
Thread-Topic: how to debug solr performance degradation
Thread-Index: AQHQUG3EGFIGDcaQ30Got4Q5AnECqp0AP49V
Message-ID: 
 <2E6A89A648463A4EBF093A9062C16683062A7CE39226@SBMAILBOX1.sb.statsbiblioteket.dk>
References: <D1121A6A.126CE%rebecca.tang@ucsf.edu>
In-Reply-To: <D1121A6A.126CE%rebecca.tang@ucsf.edu>
Accept-Language: en-US, da-DK
Content-Language: en-GB
acceptlanguage: en-US, da-DK
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Tang, Rebecca [Rebecca.Tang@ucsf.edu] wrote:
[12-15 second response time instead of 0-3]
> Solr index size 183G
> Documents in index 14364201
> We just have single solr box
> It has 100G memory
> 500G Harddrive
> 16 cpus

The usual culprit is memory (if you are using spinning drive as your storag=
e). It appears that you have enough raw memory though. Could you check how =
much memory the machine has free for disk caching? If it is a relative smal=
l amount, let's say below 50GB, then please provide a breakdown of what the=
 memory is used for (very large JVM heap for example).

> I want to pinpoint where the performance issue is coming from.  Could I h=
ave some suggestions/help on how to benchmark/debug solr performance issues=
.

Rough checking of IOWait and CPU load is a fine starting point. If if is CP=
U load then you can turn on debug in Solr admin, which should tell you wher=
e the time is spend resolving the queries. It it is IOWait then ensure a lo=
t of free memory for disk cache and/or improve your storage speed (SSDs ins=
tead of spinning drives, local storage instead of remote).

- Toke Eskildsen, State and University Library, Denmark.