lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zheng Lin Edwin Yeo <edwinye...@gmail.com>
Subject Re: Solr 6.5.1 crashing when too many queries with error or high memory usage are queried
Date Mon, 10 Jul 2017 02:59:39 GMT
I have found that it could be likely due to the hashJoin in the streaming
expression, as this will store all tuples in memory?

I have more than 12 million in the collections which I am querying, in 1
shard. The index size of the collection is 45 GB.
Physical RAM of server: 384 GB
Java Heap: 22 GB
Typical search latency: 2 to 4 seconds

Regards,
Edwin


On 7 July 2017 at 16:46, Jan Høydahl <jan.asf@cominvent.com> wrote:

> You have not told us how many documents you have, how many shards, how big
> the docs are, physical RAM, Java heap, what typical search latency is etc.
>
> If you have tried to squeeze too many docs into a single node it might get
> overloaded faster, thus sharding would help.
> If you return too much content (large fields that you won’t use) that may
> lower the max QPS for a node, so check that.
> If you are not using DocValues, faceting etc will take too much memory,
> but since you use streaming I guess you use Docvalues.
> There are products that you can put in front of Solr that can do rate
> limiting for you, such as https://getkong.org/ <https://getkong.org/>
>
> You really need to debug what is the bottleneck in your case and try to
> fix that.
>
> Can you share your key numbers here so we can do a qualified guess?
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
> > 2. jul. 2017 kl. 09.00 skrev Zheng Lin Edwin Yeo <edwinyeozl@gmail.com>:
> >
> > Hi,
> >
> > I'm currently facing the issue whereby the Solr crashed when I have
> issued
> > too many queries with error or those with high memory usage, like JSON
> > facet or Streaming expressions.
> >
> > What could be the issue here?
> >
> > I'm using Solr 6.5.1
> >
> > Regards,
> > Edwin
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message