cxf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Glynn, Eoghan" <eoghan.gl...@iona.com>
Subject RE: Problems with Chunking
Date Tue, 20 Feb 2007 16:19:01 GMT

 
> Are the redirect/authentication cases in particular HTTP 
> server bugs or limitations of HTTP? 
>
> It sounds like the later. 

Well, you could call the 401/30x response codes either a feature or
limitation of HTTP, depending on your point of view.

> I suppose we could keep a list of Servers that we should 
> should default to non-chunked, but it sounds like that 
> doesn't help the other cases.
> 
> How about this counter-counter proposal :-) It seems we have 
> a lot of cases which actually require non-chunked requests:
> - broken servers
> - authentication
> - redirects

While superficially these scenarios look similar, if we attempt to solve
the problem adaptively there are two crucial differences that become
apparent:

1. We might infer that a single 401 is possible/probable, and hence
buffer only those initial outgoing request(s) up to the receipt of the
401, before reverting to chunked. Hence the initial cost of buffering
the requests (often just one), could be amortized over many subsequent
unbuffered requests. But to avoid the ASP.NET bug, we cannot *ever*
fallback to chunking (hence my objection to your idea to buffer only up
to the first 100k).

2. While we can take an educated guess in the HTTPConduit that a 401
*may* occur (e.g. the user has gone to the trouble of configing a
AuthorizationPolicy so obviously they're expecting a challenge), but we
never know for sure when or if a 401 or redirect will be received. On
the other hand, if the server-stack support for incoming chunks is
broke, its not going to magically fix itself in the middle of a sequence
of requests.

So it seems two different adaptive strategies might be appropriate here
... 

For 401/30x: apply a heuristic to decide if a 401/30x is likely, and if
so buffer up outgoing requests until a 401/30x is received, then
transparently resend and revert to chunking. If we wanna get fancy,
allow for multiple 401/30x (e.g. a redirect followed by a challenge). So
we're unsure upfront, thus we temporarily disable chunking for this
target endpoint.
 
For broken servers: probe upfront with an innocuous GET to see if the
server barfs, and if so *permanently* turn off chunking. So we can be
sure upfront, and then permanently disable chunking for this target
endpoint.

> So why not turn off chunking by default and put in a log 
> message which states something to the extent of: "HTTP 
> chunking is turned off by default for compatability reasons. 
> For possible performance improvements, try enabling chunking."

Why turn off an optimization by default to facilitate working around a
bug in a single server-stack? 

If we don't think its worth the trouble to use an adaptive approach as
outlined above, why not just add a note to the docco ... "there are know
problems with ASP.NET consuming the chunked transfer-encoding, which can
be worked around by disabling chunking with the following configuration
fragment ..."?

Then clients of ASP.net can work around the server bug, while clients of
any other server-stack can continue to get the benefit of chunked
requests without having to change their config.
 
> For small requests (i.e. a couple K, and the most common), 
> its likely to be the same performance as woodstox wraps the 
> outputstream in a BufferedOutputStream.  Is performance the 
> only reason you want it turned on by default?

Yep, performance for large outgoing requests is the concern. 

Cheers,
Eoghan


 
> Regards,
> 
> - Dan
> 
> --
> Dan Diephouse
> Envoi Solutions
> http://envoisolutions.com | http://netzooid.com/blog
> 

Mime
View raw message