lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis.gospodne...@gmail.com>
Subject Re: Strange behaviour when tuning the caches
Date Tue, 03 Jun 2014 15:40:39 GMT
Hi,

Have you seen https://wiki.apache.org/solr/CollapsingQParserPlugin ?  May
help with the field collapsing queries.

Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/


On Tue, Jun 3, 2014 at 8:41 AM, Jean-Sebastien Vachon <
jean-sebastien.vachon@wantedanalytics.com> wrote:

> Hi Otis,
>
> We saw some improvement when increasing the size of the caches. Since
> then, we followed Shawn advice on the filterCache and gave some additional
> RAM to the JVM in order to reduce GC. The performance is very good right
> now but we are still experiencing some instability but not at the same
> level as before.
> With our current settings the number of evictions is actually very low so
> we might be able to reduce some caches to free up some additional memory
> for the JVM to use.
>
> As for the queries, it is a set of 5 million queries taken from our logs
> so they vary a lot. All I can say is that all queries involve either
> grouping/field collapsing and/or radius search around a point. Our largest
> customer is using a set of 8-10 filters that are translated as fq
> parameters. The collection contains around 13 million documents distributed
> on 5 shards with 2 replicas. The second collection has the same
> configuration and is used for indexing or as a fail-over index in case the
> first one falls.
>
> We`ll keep making adjustments today but we are pretty close of having
> something that performs while being stable.
>
> Thanks all for your help.
>
>
>
> > -----Original Message-----
> > From: Otis Gospodnetic [mailto:otis.gospodnetic@gmail.com]
> > Sent: June-03-14 12:17 AM
> > To: solr-user@lucene.apache.org
> > Subject: Re: Strange behaviour when tuning the caches
> >
> > Hi Jean-Sebastien,
> >
> > One thing you didn't mention is whether as you are increasing(I assume)
> > cache sizes you actually see performance improve?  If not, then maybe
> there
> > is no value increasing cache sizes.
> >
> > I assume you changed only one cache at a time? Were you able to get any
> > one of them to the point where there were no evictions without things
> > breaking?
> >
> > What are your queries like, can you share a few examples?
> >
> > Otis
> > --
> > Performance Monitoring * Log Analytics * Search Analytics Solr &
> > Elasticsearch Support * http://sematext.com/
> >
> >
> > On Mon, Jun 2, 2014 at 11:09 AM, Jean-Sebastien Vachon < jean-
> > sebastien.vachon@wantedanalytics.com> wrote:
> >
> > > Thanks for your quick response.
> > >
> > > Our JVM is configured with a heap of 8GB. So we are pretty close of
> > > the "optimal" configuration you are mentioning. The only other
> > > programs running is Zookeeper (which has its own storage device) and a
> > > proprietary API (with a heap of 1GB) we have on top of Solr to server
> our
> > customer`s requests.
> > >
> > > I will look into the filterCache to see if we can better use it.
> > >
> > > Thanks for your help
> > >
> > > > -----Original Message-----
> > > > From: Shawn Heisey [mailto:solr@elyograg.org]
> > > > Sent: June-02-14 10:48 AM
> > > > To: solr-user@lucene.apache.org
> > > > Subject: Re: Strange behaviour when tuning the caches
> > > >
> > > > On 6/2/2014 8:24 AM, Jean-Sebastien Vachon wrote:
> > > > > We have yet to determine where the exact breaking point is.
> > > > >
> > > > > The two patterns we are seeing are:
> > > > >
> > > > > -          less cache (around 20-30% hit/ratio), poor performance
> but
> > > > > overall good stability
> > > >
> > > > When caches are too small, a low hit ratio is expected.  Increasing
> > > > them
> > > is a
> > > > good idea, but only increase them a little bit at a time.  The
> > > filterCache in
> > > > particular should not be increased dramatically, especially the
> > > > autowarmCount value.  Filters can take a very long time to execute,
> > > > so a
> > > high
> > > > autowarmCount can result in commits taking forever.
> > > >
> > > > Each filter entry can take up a lot of heap memory -- in terms of
> > > > bytes,
> > > it is
> > > > the number of documents in the core divided by 8.  This means that
> > > > if the core has 10 million documents, each filter entry (for JUST
> > > > that
> > > > core) will take over a megabyte of RAM.
> > > >
> > > > > -          more cache (over 90% hit/ratio), improved performance
> but
> > > > > almost no stability. In that case, we start seeing messages such
> > > > > as "No shards hosting shard X" or "cancelElection did not find
> > > > > election node to remove"
> > > >
> > > > This would not be a direct result of increasing the cache size,
> > > > unless
> > > perhaps
> > > > you've increased them so they are *REALLY* big and you're running
> > > > out of RAM for the heap or OS disk cache.
> > > >
> > > > > Anyone, has any advice on what could cause this? I am beginning to
> > > > > suspect the JVM version, is there any minimal requirements
> > > > > regarding the JVM?
> > > >
> > > > Oracle Java 7 is recommended for all releases, and required for Solr
> > > 4.8.  You
> > > > just need to stay away from 7u40, 7u45, and 7u51 because of bugs in
> > > > Java itself.  Right now, the latest release is recommended, which is
> 7u60.
> > >  The
> > > > 7u21 release that you are running should be perfectly fine.
> > > >
> > > > With six 9.4GB cores per node, you'll achieve the best performance
> > > > if you have about 60GB of RAM left over for the OS disk cache to use
> > > > -- the
> > > size of
> > > > your index data on disk.  You did mention that you have 92GB of RAM
> > > > per node, but you have not said how big your Java heap is, or
> > > > whether there
> > > is
> > > > other software on the machine that may be eating up RAM for its heap
> > > > or data.
> > > >
> > > > http://wiki.apache.org/solr/SolrPerformanceProblems
> > > >
> > > > Thanks,
> > > > Shawn
> > > >
> > > > -----
> > > > Aucun virus trouvé dans ce message.
> > > > Analyse effectuée par AVG - www.avg.fr
> > > > Version: 2014.0.4570 / Base de données virale: 3950/7571 - Date:
> > > > 27/05/2014
> > >
> >
> > -----
> > Aucun virus trouvé dans ce message.
> > Analyse effectuée par AVG - www.avg.fr
> > Version: 2014.0.4570 / Base de données virale: 3950/7571 - Date:
> > 27/05/2014 La Base de données des virus a expiré.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message