hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gordon Mohr (JIRA)" <j...@apache.org>
Subject [jira] Created: (HTTPCLIENT-566) HttpParser triggers unfriendly OutOfMemoryError on challenging input
Date Mon, 15 May 2006 20:34:07 GMT
HttpParser triggers unfriendly OutOfMemoryError on challenging input

         Key: HTTPCLIENT-566
         URL: http://issues.apache.org/jira/browse/HTTPCLIENT-566
     Project: Jakarta HttpClient
        Type: Bug

  Components: HttpClient  
    Versions: 3.0.1    
    Reporter: Gordon Mohr

Many users of HttpClient use it to connect to servers which generate challenging HTTP responses,
such as responses which include an arbitrarily large number of headers or headers of arbitrarily
large size. (Sometimes such headers are conformant with the spec, in that they contain legal
characters in plausible header formats; other times these are filled with binary content that
is a violation of the relevant specs. Even when technically legal, often such giant headers
are the inadvertent result of server-side bugs.)

As a Java execution environment always has a hard cap on the available heap space, any parsing
code which can use an arbitrary amount of memory risks triggering an OutOfMemoryError, either
in its own thread or even another thread that happens to need memory after the parsing thread
has exhausted it all. 

Such OutOfMemoryErrors are a particularly unfriendly way to indicate that a practical limit
has been exceeded, compared to other options. They can hide the thread of execution which
is most to blame. It is hard and awkward to set up handlers that catch and recover from OOMEs
wherever they are most likely to occur.  Even with such handlers, the actual allocation triggering
an OOME may occur in another critical thread, even if that thread has minimal and well-controlled
memory needs. 

HttpClient ought to provide one or more ways for a user to protect against such OOMEs, and
instead receive a more convenient/recoverable indication of an HTTP response that is impossible
to process with the HttpClient library within the available resources. Many approaches are
possible; the easiest would be to allow a user of HttpClient to set their own optional, pragmatic
limits on header sizes and number. Then, just as a user may already cleanly cancel the stream-reading
of an arbitrarily-long content-body without fouling up their application state, they would
be able to cancel the parsing of oversized response headers. 

Similar issues have been discussed before, for example in Bugzilla bug #25468 (http://issues.apache.org/bugzilla/show_bug.cgi?id=25468)
which was to "Provide HttpParser plug-in mechanism." Though that issue is marked resolved/fixed,
there is no such plug-in mechanism allowing an OOME workaround in the 3.x HttpClient, and
it is not clear that a mechanism/work-around exists in whatever 4.0 work has been completed.

So my suggestion is that this new issue be used to uniquely track the OOME risk in HttpParser,
and only be considered "fixed" when some version of HttpClient offers an alternative to throwing
OOMEs as a way of dealing with challenging HTTP responses. Alternatively, this could simply
become the issue in the new system for collecting user-contributed workarounds/patches. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
For more information on JIRA, see:

To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org

View raw message