lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Susheel Kumar <susheel2...@gmail.com>
Subject Re: How to stop long running/memory eating query
Date Thu, 17 Nov 2016 16:39:24 GMT
Hi Erick, you got it.  I missed to put the rest of the query and the
parameter which caused the issue is start parameter.  The start parameter
for this query was put like 30+ milllion by the user due to bad UI design
(deep pagination issue) and bringing the whole cluster down .

Thnx

On Thu, Nov 17, 2016 at 11:08 AM, Erick Erickson <erickerickson@gmail.com>
wrote:

> That query frankly doesn't seem like it'd lead to OOM or run for a
> very long time unless there are (at least) hundreds of terms and a
> _lot_ of documents. Or you're trying to return a zillion rows. Or
> you're faceting on a high cardinality field. Or....
>
> The terms should be being kept in MMapDirectory space.
>
> My guess is that you aren't showing the part that's really causing the
> problem, perhaps try peeling parts of the query off until you find the
> culprit?
>
> And if you're sorting, faceting or the like docValues will help
> prevent OOM problems.
>
> Best,
> Erick
>
> On Thu, Nov 17, 2016 at 7:17 AM, Davis, Daniel (NIH/NLM) [C]
> <daniel.davis@nih.gov> wrote:
> > Mikhail,
> >
> > If the query is not asynchronous, it would certainly be OK to stop the
> long-running query if the client socket is disconnected.   I know that is a
> feature of the niche indexer used in the products of www.indexengines.com,
> because I wrote it.   We did not have asynchronous queries, and because of
> the content and query-time deduplication, some queries could take hours
> -that's 72 billion objects on a 2U box for you.   Hope they've added better
> index-time deduplication by now.
> >
> > Thanks,
> >
> > -dan
> >
> > -----Original Message-----
> > From: Mikhail Khludnev [mailto:mkhl@apache.org]
> > Sent: Thursday, November 17, 2016 6:55 AM
> > To: solr-user <solr-user@lucene.apache.org>
> > Subject: Re: How to stop long running/memory eating query
> >
> > There is a circuit breaker
> > https://cwiki.apache.org/confluence/display/solr/
> Common+Query+Parameters#CommonQueryParameters-ThetimeAllowedParameter
> > If I'm right, it does not interrupt faceting.
> >
> > On Thu, Nov 17, 2016 at 2:07 PM, Susheel Kumar <susheel2777@gmail.com>
> > wrote:
> >
> >> Hello,
> >>
> >> We found a query which was running forever and thus causing OOM (
> >> q=+AND++AND+Tom+AND+Jerry...).  Is there any way similar to SQL/No SQL
> >> world where we can watch currently executed queries and able to kill
> them.
> >> This can be desiring feature in these situations and avoid whole
> >> cluster going down. Is there any existing JIRA/can create one.
> >>
> >> Also what would be the different ways we can examine and stop such
> >> queries to execute.
> >>
> >> Thanks,
> >> Susheel
> >>
> >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message