hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: Handling idle timeout disconnect
Date Mon, 24 Feb 2014 08:57:21 GMT
On Mon, 2014-02-24 at 00:40 +0000, sebb wrote:
> On 22 February 2014 17:23, sebb <sebbaz@gmail.com> wrote:
> > On 22 February 2014 16:37, Oleg Kalnichevski <olegk@apache.org> wrote:
> >> On Sat, 2014-02-22 at 14:52 +0000, sebb wrote:
> >>
> >> ...
> >>
> >>> > IIRC, we disabled retries because they were causing unexpected extra
> >>> > traffic to be sent to servers.
> >>> > Also, generally testers need to know when requests fail, as it affects
> >>> > throughput.
> >>> >
> >>> > So we may need our own retry handler for this specific case.
> >>>
> >>> Actually, it looks like there may be a bug in the
> >>> DefaultHttpRequestRetryHandler class which is why it was not suitable
> >>> for JMeter.
> >>>
> >>> We invoke it with the parameter requestSentRetryEnabled = false.
> >>>
> >>> However we found that additional requests were being seen at the
> >>> server, so we now default to zero retries.
> >>>
> >>> I've just had a look at the code, and the parameter
> >>> requestSentRetryEnabled seems to be applied too late.
> >>> It is checked AFTER the check for idempotent requests.
> >>>
> >>> So a GET will be resent even if requestSentRetryEnabled is false.
> >>> That does not seem right.
> >>>
> >>> Shall I raise a JIRA?
> >>>
> >>
> >> DefaultHttpRequestRetryHandler implementation goes back to the dark ages
> >> of HttpClient 2.x, but I think it was always intended to work that way.
> >> So, it is a feature.
> >
> > I've also now checked the User Guide, which says:
> >
> > * HttpClient will automatically retry those methods that are assumed
> > to be idempotent.
> > * HttpClient will automatically retry those methods that fail with a
> > transport exception while the HTTP request is still being transmitted
> > to the target server (i.e. the request has not been fully transmitted
> > to the server).
> >
> > That is true, but does not mention the requestSentRetryEnabled flag
> >
> > Currently, DefaultHttpRequestRetryHandler Javadoc says:
> >
> > requestSentRetryEnabled true if it's OK to retry requests that have been sent
> >
> > Which is a bit misleading, as it only applies to non-idempotent
> > requests - idempotent requests are always retried.
> >
> > So I think the current Javadoc could do with updating to avoid giving
> > the impression that the flag can be used to prevent idempotent
> > retries.
> >
> >> I am fine with changing the behavior, but in 4.4 branch.
> >
> > That would also need reflecting in the User Guide.
> > And we would need to decide if retries should be separately enabled
> > for idempotent requests.
> >
> > In the meantime, JMeter can create its own retry handler.
> 
> Or so I thought ...
> 
> Unfortunately it appears that the disconnect only occurs - or perhaps
> is only detected - after the next request has been fully sent.
> 
> This means that POST requests that fall foul of this won't be retried
> anyway, and cannot be safely retried.
> 
> What would perhaps work for JMeter would be a version of the stale
> connection check that was conditionally applied to a connection.
> For example only after the connection has been idle for a while.
> Unfortunately unconditional stale checking affects performance too
> much.
> 
> The only other solution I can see at present is to be able to somehow
> detect the disconnect earlier.

Sadly, this just does not seem to work well with Java classic (blocking)
I/O model. We, as a project, have been battling this problem for over 10
years as far as I remember. Sometimes small requests can fully written
into some kind of internal output buffer even though the underlying
connection is no longer valid. I/O exception is thrown only on the
subsequent read attempt.


> For example after sending the first line of the request, would it be
> possible to somehow check if the connection has been dropped?
> And could this be done without compromising the performance?
> 

It would hardly be any different than the stale check used presently.

Oleg


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Mime
View raw message