lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Schultz <ch...@christopherschultz.net>
Subject Re: Solr Server crashes when requesting a result with too large resultRows
Date Tue, 31 Jul 2018 20:40:08 GMT
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Georg,

On 7/31/18 12:33 PM, Georg Fette wrote:
> Yes ist is only one of the processors that is at maximum capacity.

Ok.

> How do I do something like a thread-dump of a single thread ?

Here's how to get a thread dump of the whole JVM:
https://wiki.apache.org/tomcat/HowTo#How_do_I_obtain_a_thread_dump_of_my
_running_webapp_.3F

The "tid" field of each thread is usually the same as the process-id
from a "top" or "ps" listing, except it's often shown in hex instead
of decimal.

Have a look at this for some guidance:
http://javadrama.blogspot.com/2012/02/why-is-java-eating-my-cpu.html

Some tools dump the tid in hex, others in decimal. It's frustrating
sometimes.

> We run the Solr from the command line out-of-the-box and not in a 
> code development environment. Are there parameters that can be 
> configured so that the server creates dumps ?
You don't want this to happen automatically. Instead, you'll want to
trigger a dump manually for debugging purposes.

- -chris


> Am 31.07.2018 um 15:07 schrieb Christopher Schultz: Georg,
> 
> On 7/31/18 4:39 AM, Georg Fette wrote:
>>>> We run the server version 7.3.1. on a machine with 32GB RAM
>>>> in a mode having -10g.
>>>> 
>>>> When requesting a query with
>>>> 
>>>> q={!boost 
>>>> b=sv_int_catalog_count_document}string_catalog_aliases:(*2*)&fq=str
ing
>
>>>> 
_field_type:catalog_entry&rows=2147483647
>>>> 
>>>> 
>>>> the server takes all available memory up to 10GB and is then
>>>> no longer accessible with one processor at 100%.
> Is it a single thread which takes the CPU or more than one? Can
> you identify that thread and take a thread dump to get a backtrace
> for that thread?
> 
>>>> When we reduce the rows parameter to 10000000 the query
>>>> works. The query returns only 581 results.
>>>> 
>>>> The documentation at 
>>>> https://wiki.apache.org/solr/CommonQueryParameters states
>>>> that as the "rows" parameter a "ridiculously large value" may
>>>> be used, but this could pose a problem. The number we used
>>>> was Int.max from Java.
> Interesting. I wonder if Solr attempts to pre-allocate a result 
> buffer. Requesting 2147483647 rows can have an adverse affect on
> most pre-allocated data structures.
> 
> -chris
>> 
> 
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAltgySgACgkQHPApP6U8
pFjgKxAAxfbUmcj81+CpmTwHaPsz8Zb70HX4o/1eDGwALMhuvg8MyTaZnR9rSPy3
LHhAn0dtdnhp7Pe3NWRrYFdzKOZjQ85jiEcW96bzCe5ggJmnvs9a9VeEJ+5b4AXN
XMtSMo8Ph7BvAWeTQcwmsiK8w2grAzaV6zXEetxaXgL0+16wfIjyNBteiQHkpcjo
T5T5UzSzwyuAxFJkxSdbsF6SAJD7+zwbOEUQlURlUBsmzgam124ojgNl3gEG8d/V
SSFhI1vnuj7pkdFLSZm7BDdAw6KjnOeM3yE3VKh5Lem4CRNLrP3ZvKrzKVlWTFJ4
dAIuJL6GUSMEFU0MCwQZjFxmtWNMwl/MIdDD8Yp9m/GislLXbcOi4oBbmWTNnuqU
SPtmjdV+7fcIRl8AWc0bzLbK4nFYlVFzhiijR5am+pvF13TB/WQ8eOn9uifSPxWb
OHzrU+fMV0fvIe5pZxqkcHEBas5QiZKZ5yH6Zz+xLldF4nh9Q4A6CJu/21qU/Kxd
Dp2lenZEjKc90FKpSVMXqxJNM0n7geRmTSgv8imeoQf5+H6VU7dll1xGQkTnXtR9
UyV/U1fj12z2UjzcY6ePuJ8BadIx+cSf6H3q4bcJOGZ884lI+bDX08C/89hb/5vT
2NE5+tK1jAOX/ESClb6eFFMcJzBww/CoIxb9PpRqgw3HJKYuVpY=
=mS/y
-----END PGP SIGNATURE-----

Mime
View raw message