lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis.gospodne...@gmail.com>
Subject Re: Strange behaviour when tuning the caches
Date Tue, 03 Jun 2014 04:17:22 GMT
Hi Jean-Sebastien,

One thing you didn't mention is whether as you are increasing(I assume)
cache sizes you actually see performance improve?  If not, then maybe there
is no value increasing cache sizes.

I assume you changed only one cache at a time? Were you able to get any one
of them to the point where there were no evictions without things breaking?

What are your queries like, can you share a few examples?

Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/


On Mon, Jun 2, 2014 at 11:09 AM, Jean-Sebastien Vachon <
jean-sebastien.vachon@wantedanalytics.com> wrote:

> Thanks for your quick response.
>
> Our JVM is configured with a heap of 8GB. So we are pretty close of the
> "optimal" configuration you are mentioning. The only other programs running
> is Zookeeper (which has its own storage device) and a proprietary API (with
> a heap of 1GB) we have on top of Solr to server our customer`s requests.
>
> I will look into the filterCache to see if we can better use it.
>
> Thanks for your help
>
> > -----Original Message-----
> > From: Shawn Heisey [mailto:solr@elyograg.org]
> > Sent: June-02-14 10:48 AM
> > To: solr-user@lucene.apache.org
> > Subject: Re: Strange behaviour when tuning the caches
> >
> > On 6/2/2014 8:24 AM, Jean-Sebastien Vachon wrote:
> > > We have yet to determine where the exact breaking point is.
> > >
> > > The two patterns we are seeing are:
> > >
> > > -          less cache (around 20-30% hit/ratio), poor performance but
> > > overall good stability
> >
> > When caches are too small, a low hit ratio is expected.  Increasing them
> is a
> > good idea, but only increase them a little bit at a time.  The
> filterCache in
> > particular should not be increased dramatically, especially the
> > autowarmCount value.  Filters can take a very long time to execute, so a
> high
> > autowarmCount can result in commits taking forever.
> >
> > Each filter entry can take up a lot of heap memory -- in terms of bytes,
> it is
> > the number of documents in the core divided by 8.  This means that if the
> > core has 10 million documents, each filter entry (for JUST that
> > core) will take over a megabyte of RAM.
> >
> > > -          more cache (over 90% hit/ratio), improved performance but
> > > almost no stability. In that case, we start seeing messages such as
> > > "No shards hosting shard X" or "cancelElection did not find election
> > > node to remove"
> >
> > This would not be a direct result of increasing the cache size, unless
> perhaps
> > you've increased them so they are *REALLY* big and you're running out of
> > RAM for the heap or OS disk cache.
> >
> > > Anyone, has any advice on what could cause this? I am beginning to
> > > suspect the JVM version, is there any minimal requirements regarding
> > > the JVM?
> >
> > Oracle Java 7 is recommended for all releases, and required for Solr
> 4.8.  You
> > just need to stay away from 7u40, 7u45, and 7u51 because of bugs in Java
> > itself.  Right now, the latest release is recommended, which is 7u60.
>  The
> > 7u21 release that you are running should be perfectly fine.
> >
> > With six 9.4GB cores per node, you'll achieve the best performance if you
> > have about 60GB of RAM left over for the OS disk cache to use -- the
> size of
> > your index data on disk.  You did mention that you have 92GB of RAM per
> > node, but you have not said how big your Java heap is, or whether there
> is
> > other software on the machine that may be eating up RAM for its heap or
> > data.
> >
> > http://wiki.apache.org/solr/SolrPerformanceProblems
> >
> > Thanks,
> > Shawn
> >
> > -----
> > Aucun virus trouvé dans ce message.
> > Analyse effectuée par AVG - www.avg.fr
> > Version: 2014.0.4570 / Base de données virale: 3950/7571 - Date:
> > 27/05/2014
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message