hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: BasicAsyncResponseConsumer needs OOM protection?
Date Fri, 03 Jun 2016 18:45:14 GMT
On Thu, 2016-06-02 at 19:35 +0000, Idzerda, Edan wrote:
> > -----Original Message-----
> > From: Oleg Kalnichevski [mailto:olegk@apache.org]
> > Sent: Tuesday, May 31, 2016 3:04 PM
> > To: HttpComponents Project <dev@hc.apache.org>
> > Subject: Re: BasicAsyncResponseConsumer needs OOM protection?
> > 
> > On Tue, 2016-05-31 at 16:03 +0000, Idzerda, Edan wrote:
> > > Using this patch, I can request HTTP responses that exceed available
> > memory without resulting in a hang.  Does this seem like the appropriate
> > place to patch this error?  Or should I create a JIRA against this issue and
> > suggest this patch as a solution?  I've attached a diff to this email
> > >
> > > Thanks for your assistance and for working on such a great set of Java
> > libraries.
> > 
> > Edan
> > 
> > Basic request / response consumers are not supposed to be used for
> > anything other than the most basic use cases. One is generally advised
> > to build custom response consumers specifically tailed to their
> > particular application.
> 
> I understand that, and thanks for the other examples. For what it's worth, I have load
tested the reverse proxy using the BasicAsyncResponseConsumer and it performs extremely well
with small messages.
> 
> However, if a developer uses the HttpAsyncClient without specifying a specific response
consumer, they are using this Basic one by default.  If they happen to experience an OOM condition
because their heap cannot handle a response size, the NIO Connection Pool cannot make any
new requests. I found this difficult to debug because I assumed the connection pool would
recover from this unexpected condition.
> 
> Isn't the AbstractMultiworkerIOReactor expecting that any error like this would be captured
by the response consumer?   If that's true, shouldn't the BasicAsyncResponseConsumer suppress
a hard error like OOM to prevent this problem?  Or should the IOReactor layer recover better?
> 

Not if it is a subclass of java.lang.Error. Errors represent abnormal,
fatal, non-recoverable conditions and usually should cause JRE to
terminate. 

If in the context of your application OOM errors can be considered
recoverable and can re-thrown as I/O exceptions it is your decision to
make but this may not necessarily be the case for most users.

Oleg


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Mime
View raw message