Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
MIME-Version: 1.0
Date: Thu, 27 Aug 2009 15:06:55 +0100
Message-ID: <69a99bd60908270706w6dbf25es59b7be88a09bec56@mail.gmail.com>
Subject: Optimal Cache Settings, complicated by regular commits
From: Andrew Ingram <andy@andrewingram.net>
To: solr-user@lucene.apache.org
Content-Type: multipart/alternative; boundary=0015174c1cfc8842f404722014a4

--0015174c1cfc8842f404722014a4
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Hi all,
I'm trying to work out the optimum cache settings for our Solr server, I'll
begin by outlining our usage.

Number of documents: approximately 25,000
Commit frequency: sometimes we do massive amounts of sequential commits,
most of the time its less frequent but still several times an hour
We make heavy use of faceting and sorting, and the number of possible facets
led to choosing a filterCache size of about 50,000

The problem we have is that the default cache settings resulting in very low
hit rates (less than 30% for documents, less than 1% for filterCache), so we
upped the cache size up gradually until the hit rates were in the 80s-90s,
now we have the issue of commits being very slow (more than 5 seconds for a
document), to the point where it causes a timeout elsewhere in our systems.
This is made worse by the fact that committing seems to empty the cache,
given that it takes about an hour to get the cache to a good state this is
obviously very problematic.

Is there a way for commits to selectively empty the cache? Any advice
regarding the config would be appreciated. The server load is relatively
low, ideally we're looking to minimize the response time rather than aim for
CPU or memory efficiency.

Regards,
Andrew Ingram

--0015174c1cfc8842f404722014a4--