lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joel Bernstein <joels...@gmail.com>
Subject Re: Slowly running OOM due to Query instances?!
Date Fri, 07 Jul 2017 20:41:06 GMT
I suspect this is related to caching in some way, possibly one of the
following:

1) You have large queries in a large cache.
2) Your custom query parsers have a bug that is causing a leak with the
caches.

Joel Bernstein
http://joelsolr.blogspot.com/

On Fri, Jul 7, 2017 at 9:41 AM, Markus Jelsma <markus.jelsma@openindex.io>
wrote:

> Hello,
>
> Sorry, i forgot to mention we already run 6.6.
>
> Now i am looking at the sampler again on a freshly restarted instance.
> This node has almost 6000 uncollectable TermQuery instances at this moment.
> There are just 3.4 queries per second on this node. The final query is
> complex but i cannot understand why so many TermQuery. Is this even normal?
> I haven't seen this before, but i also have to say i never looked at this
> specific thing before.
>
> Thanks,
> Markus
>
>
> -----Original message-----
> > From:Erik Hatcher <erik.hatcher@gmail.com>
> > Sent: Friday 7th July 2017 15:11
> > To: solr-user@lucene.apache.org
> > Subject: Re: Slowly running OOM due to Query instances?!
> >
> > With generated Query’s, one has to be really careful with .equals and
> .hashCode implementations.  That may not be applicable here, but something
> that has bitten me with caching.   Note that there were fixes made in Solr
> 6.6 with PayloadScoreQuery in this regard. See LUCENE-7808 and LUCENE-7481
> >
> >       Erik
> >
> >
> > > On Jul 7, 2017, at 7:01 AM, Markus Jelsma <markus.jelsma@openindex.io>
> wrote:
> > >
> > > Hello,
> > >
> > > This morning i spotted our QTime suddenly go up. This has been going
> on for a few hours by now and coincides with a serious increase in heap
> consumption. No node ran out of memory so far but either that is going to
> happen soon, or the nodes become unusable in another manner.
> > >
> > > I restarted one of the Solr instances and launched VisualVM at it, and
> some other nodes that use to much heap. Starting the memory sampler,
> something was obvious straight away.
> > >
> > > The nodes consuming too much heap all have a serious amount of *Query,
> and BooleanClause instances, PayloadScoreQuery, TermQuery, BoostQuery,
> BooleanQuery, SpanTermQuery and so forth. Lots of Builder and Term
> instances too, very distinct from the node that was just freshly restarted.
> > >
> > > Another peculiarity, some nodes have exactly 65536 instances of
> TermQuery and/or BoostQuery, probably unrelated but not something i would
> have expected to see anyway.
> > >
> > > So, what's up? We do have a custom query parser extending
> EdismaxQParser, it transliterates dates and creates payload and span
> queries. I may be doing something wrong but i don't know, i have made and
> used a variety of QParsers, for many years but this is new. Any hints on
> where to look, what to watch out for?
> > >
> > > Many thanks!
> > > Markus
> > >
> > > Xmx 800m, 8 GB RAM, SSD
> > > 2 shards, three replica's
> > > replica size ~17 GB, 2.2 million docs/replica
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message