Mailing-List: contact dev-help@hc.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "HttpComponents Project" <dev@hc.apache.org>
MIME-Version: 1.0
In-Reply-To: <1464979514.4082.7.camel@apache.org>
References: <eeee0360266a41d984f6036bdb99e4cc@c3piexchange1.corp.premierinc.com>
 <1464721441.20289.5.camel@apache.org> <1b08bbb964a64010bb0a5779b0611c69@c3piexchange1.corp.premierinc.com>
 <1464979514.4082.7.camel@apache.org>
From: sebb <sebbaz@gmail.com>
Date: Sat, 4 Jun 2016 14:14:35 +0100
Message-ID: <CAOGo0VbtnXCBnOAED7105Ls8uN-aZpMwQZKbAAnBidUsuqy=cA@mail.gmail.com>
Subject: Re: BasicAsyncResponseConsumer needs OOM protection?
To: HttpComponents Project <dev@hc.apache.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
archived-at: Sat, 04 Jun 2016 13:14:59 -0000

On 3 June 2016 at 19:45, Oleg Kalnichevski <olegk@apache.org> wrote:
> On Thu, 2016-06-02 at 19:35 +0000, Idzerda, Edan wrote:
>> > -----Original Message-----
>> > From: Oleg Kalnichevski [mailto:olegk@apache.org]
>> > Sent: Tuesday, May 31, 2016 3:04 PM
>> > To: HttpComponents Project <dev@hc.apache.org>
>> > Subject: Re: BasicAsyncResponseConsumer needs OOM protection?
>> >
>> > On Tue, 2016-05-31 at 16:03 +0000, Idzerda, Edan wrote:
>> > > Using this patch, I can request HTTP responses that exceed available
>> > memory without resulting in a hang.  Does this seem like the appropria=
te
>> > place to patch this error?  Or should I create a JIRA against this iss=
ue and
>> > suggest this patch as a solution?  I've attached a diff to this email
>> > >
>> > > Thanks for your assistance and for working on such a great set of Ja=
va
>> > libraries.
>> >
>> > Edan
>> >
>> > Basic request / response consumers are not supposed to be used for
>> > anything other than the most basic use cases. One is generally advised
>> > to build custom response consumers specifically tailed to their
>> > particular application.
>>
>> I understand that, and thanks for the other examples. For what it's wort=
h, I have load tested the reverse proxy using the BasicAsyncResponseConsume=
r and it performs extremely well with small messages.
>>
>> However, if a developer uses the HttpAsyncClient without specifying a sp=
ecific response consumer, they are using this Basic one by default.  If the=
y happen to experience an OOM condition because their heap cannot handle a =
response size, the NIO Connection Pool cannot make any new requests. I foun=
d this difficult to debug because I assumed the connection pool would recov=
er from this unexpected condition.
>>
>> Isn't the AbstractMultiworkerIOReactor expecting that any error like thi=
s would be captured by the response consumer?   If that's true, shouldn't t=
he BasicAsyncResponseConsumer suppress a hard error like OOM to prevent thi=
s problem?  Or should the IOReactor layer recover better?
>>
>
> Not if it is a subclass of java.lang.Error. Errors represent abnormal,
> fatal, non-recoverable conditions and usually should cause JRE to
> terminate.
>
> If in the context of your application OOM errors can be considered
> recoverable and can re-thrown as I/O exceptions it is your decision to
> make but this may not necessarily be the case for most users.

Also in general it is quite difficult to recover from OOM, as more
memory may be needed (at least temporarily).

> Oleg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org