Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of idokissos@gmail.com designates
 74.125.82.174 as permitted sender)
MIME-Version: 1.0
Sender: idokissos@gmail.com
In-Reply-To: <7FDD0AB1-545F-43F5-B4E6-9F7EDD2F19AE@wunderwood.org>
References: 
 <CABGq8w8qz1r2xSqeYKP-eUTBkT8KfD1bjyP=BrBdSgMiJwTd6A@mail.gmail.com>
 <151D4D22-0C9F-4A3F-9F82-1B3C6A5ABF5D@wunderwood.org>
 <CABGq8w_P7Fk_hJyySAWZ5ZDV-zbjYUg_pxODZ_qfYFr2oxJnkA@mail.gmail.com>
 <7FDD0AB1-545F-43F5-B4E6-9F7EDD2F19AE@wunderwood.org>
From: Manuel Le Normand <manuel.lenormand@gmail.com>
Date: Tue, 18 Jun 2013 00:45:14 +0300
Message-ID: 
 <CABGq8w-YFpD7MFiNFmO5HzC-4r5iNNZg893AUvxoxFVKMrNp9w@mail.gmail.com>
Subject: Re: Avoiding OOM fatal crash
To: "solr-user@lucene.apache.org" <solr-user@lucene.apache.org>
Content-Type: multipart/alternative; boundary=089e013d1a04c834ed04df608484

--089e013d1a04c834ed04df608484
Content-Type: text/plain; charset=ISO-8859-1

Unfortunately my organisation's too big to control or teach every employee
what are the limits, as well as they can vary (many facets - how much is
ok?, asking for too many fields in proportion of too many rows etc)

Don't you think it is preferable to "commit" the maxBufferSize in the JVM
heap for indexing only?


On Tue, Jun 18, 2013 at 12:11 AM, Walter Underwood <wunder@wunderwood.org>wrote:

> Make them aware of what is required. Solr is not designed to return huge
> requests.
>
> If you need to do this, you will need to run the JVM with a big enough
> heap to build the request. You are getting OOM because the JVM does not
> have enough memory to build a response with 100K documents.
>
> wunder
>
> On Jun 17, 2013, at 1:57 PM, Manuel Le Normand wrote:
>
> > One of my users requested it, they are less aware of what's allowed and I
> > don't want apriori blocking them for long specific request (there are
> other
> > params that might end up OOMing me).
> >
> > I thought of timeAllowed restriction, but also this solution cannot
> > guarantee during this delay I would not get the JVM heap flooded (for
> > example I already have all cashed and my RAM io's are very fast)
> >
> >
> > On Mon, Jun 17, 2013 at 11:47 PM, Walter Underwood <
> wunder@wunderwood.org>wrote:
> >
> >> Don't request 100K docs in a single query. Fetch them in smaller
> batches.
> >>
> >> wunder
> >>
> >> On Jun 17, 2013, at 1:44 PM, Manuel Le Normand wrote:
> >>
> >>> Hello again,
> >>>
> >>> After a heavy query on my index (returning 100K docs in a single query)
> >> my
> >>> JVM heap's floods and I get an JAVA OOM exception, and then that my
> >>> GCcannot collect anything (GC
> >>> overhead limit exceeded) as these memory chunks are not disposable.
> >>>
> >>> I want to afford queries like this, my concern is that this case
> >> provokes a
> >>> total Solr crash, returning a 503 Internal Server Error while trying
> to *
> >>> index.*
> >>>
> >>> Is there anyway to separate these two logics? I'm fine with solr not
> >> being
> >>> able to return any response after returning this OOM, but I don't see
> the
> >>> justification the query to flood JVM's internal (bounded) buffers for
> >>> writings.
> >>>
> >>> Thanks,
> >>> Manuel
> >>
>
>
>
>
>

--089e013d1a04c834ed04df608484--