hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sebb <seb...@gmail.com>
Subject Re: BasicAsyncResponseConsumer needs OOM protection?
Date Sat, 04 Jun 2016 13:14:35 GMT
On 3 June 2016 at 19:45, Oleg Kalnichevski <olegk@apache.org> wrote:
> On Thu, 2016-06-02 at 19:35 +0000, Idzerda, Edan wrote:
>> > -----Original Message-----
>> > From: Oleg Kalnichevski [mailto:olegk@apache.org]
>> > Sent: Tuesday, May 31, 2016 3:04 PM
>> > To: HttpComponents Project <dev@hc.apache.org>
>> > Subject: Re: BasicAsyncResponseConsumer needs OOM protection?
>> >
>> > On Tue, 2016-05-31 at 16:03 +0000, Idzerda, Edan wrote:
>> > > Using this patch, I can request HTTP responses that exceed available
>> > memory without resulting in a hang.  Does this seem like the appropriate
>> > place to patch this error?  Or should I create a JIRA against this issue and
>> > suggest this patch as a solution?  I've attached a diff to this email
>> > >
>> > > Thanks for your assistance and for working on such a great set of Java
>> > libraries.
>> >
>> > Edan
>> >
>> > Basic request / response consumers are not supposed to be used for
>> > anything other than the most basic use cases. One is generally advised
>> > to build custom response consumers specifically tailed to their
>> > particular application.
>>
>> I understand that, and thanks for the other examples. For what it's worth, I have
load tested the reverse proxy using the BasicAsyncResponseConsumer and it performs extremely
well with small messages.
>>
>> However, if a developer uses the HttpAsyncClient without specifying a specific response
consumer, they are using this Basic one by default.  If they happen to experience an OOM condition
because their heap cannot handle a response size, the NIO Connection Pool cannot make any
new requests. I found this difficult to debug because I assumed the connection pool would
recover from this unexpected condition.
>>
>> Isn't the AbstractMultiworkerIOReactor expecting that any error like this would be
captured by the response consumer?   If that's true, shouldn't the BasicAsyncResponseConsumer
suppress a hard error like OOM to prevent this problem?  Or should the IOReactor layer recover
better?
>>
>
> Not if it is a subclass of java.lang.Error. Errors represent abnormal,
> fatal, non-recoverable conditions and usually should cause JRE to
> terminate.
>
> If in the context of your application OOM errors can be considered
> recoverable and can re-thrown as I/O exceptions it is your decision to
> make but this may not necessarily be the case for most users.

Also in general it is quite difficult to recover from OOM, as more
memory may be needed (at least temporarily).

> Oleg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Mime
View raw message