hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ortwin Gl├╝ck <...@odi.ch>
Subject Re: [jira] Resolved: (HTTPCORE-3) HttpParser triggers unfriendly OutOfMemoryError on challenging input
Date Wed, 17 May 2006 07:29:09 GMT


Gordon Mohr wrote:
> If I understand the HttpCore code properly, there is no direct facility 
> for protecting against the OOME in the code -- just a chance to hook in 
> a theoretical alternate implementation that would address the problem.
> 
> Is that correct?

Yes.

> To use the HttpCore-4.0 facility, it appears I would create my own 
> HttpDataReceiver implementation which keeps a count of the bytes it 
> shovels & throws an IO or HTTP exception when some count is exceeded; 
> create a factory that makes such receivers; install that factory into 
> each HttpClientConnection instance before it begins receiving data.

Correct.

> This could work, but seems a roundabout and obscure approach. The really 
> valuable feature would be for OOME-resistance -- and friendly, usable 
> indicators that extreme content has been encountered -- to be features 
> of the library. It's require a switch or paramter to enable, rather than 
> patching in custom/third-party code.

An OOM Exception is exactly that: a friendly usable indicator that an 
extreme condition has been encountered. I am personally against another 
mechanism that tries to somehow monitor heap memory.

I agree that by sending large header information a malicious server 
could DOS a spider based on HttpCore. So it makes sense for this use 
case to include a protection parameter that sets an upper limit to the 
information in headers. A default of 100 KB should be enough for the 
real world. This is the easy solution.

Another possibility is to use a header stream like we do for the message 
body. This would mean a refactoring and probably cause painful client 
code. We would still have to protect ourselves against long header lines 
which is again not easy.

This all boils down to a missing feature: byte counting in the connection.

> Is there a summary of expected dates of Core-4.0/Client-4.0 release 
> somewhere, or any assessments of how the 4.0 codebases match up against 
> 3.0 features? (Is it reasonable for an HttpClient-3.0-using project to 
> consider transitioning to the 4.0 codebase(s)?)

This is the only plan there is:
http://wiki.apache.org/jakarta-httpclient/HttpComponentsProjectRoadmap


> - Gordon @ IA

Ortwin

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org


Mime
View raw message