hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sebb <seb...@gmail.com>
Subject Re: Handling idle timeout disconnect
Date Mon, 24 Feb 2014 11:59:23 GMT
On 24 February 2014 08:57, Oleg Kalnichevski <olegk@apache.org> wrote:
> On Mon, 2014-02-24 at 00:40 +0000, sebb wrote:
>> On 22 February 2014 17:23, sebb <sebbaz@gmail.com> wrote:
>> > On 22 February 2014 16:37, Oleg Kalnichevski <olegk@apache.org> wrote:
>> >> On Sat, 2014-02-22 at 14:52 +0000, sebb wrote:
>> >>
>> >> ...
>> >>
>> >>> > IIRC, we disabled retries because they were causing unexpected
extra
>> >>> > traffic to be sent to servers.
>> >>> > Also, generally testers need to know when requests fail, as it
affects
>> >>> > throughput.
>> >>> >
>> >>> > So we may need our own retry handler for this specific case.
>> >>>
>> >>> Actually, it looks like there may be a bug in the
>> >>> DefaultHttpRequestRetryHandler class which is why it was not suitable
>> >>> for JMeter.
>> >>>
>> >>> We invoke it with the parameter requestSentRetryEnabled = false.
>> >>>
>> >>> However we found that additional requests were being seen at the
>> >>> server, so we now default to zero retries.
>> >>>
>> >>> I've just had a look at the code, and the parameter
>> >>> requestSentRetryEnabled seems to be applied too late.
>> >>> It is checked AFTER the check for idempotent requests.
>> >>>
>> >>> So a GET will be resent even if requestSentRetryEnabled is false.
>> >>> That does not seem right.
>> >>>
>> >>> Shall I raise a JIRA?
>> >>>
>> >>
>> >> DefaultHttpRequestRetryHandler implementation goes back to the dark ages
>> >> of HttpClient 2.x, but I think it was always intended to work that way.
>> >> So, it is a feature.
>> >
>> > I've also now checked the User Guide, which says:
>> >
>> > * HttpClient will automatically retry those methods that are assumed
>> > to be idempotent.
>> > * HttpClient will automatically retry those methods that fail with a
>> > transport exception while the HTTP request is still being transmitted
>> > to the target server (i.e. the request has not been fully transmitted
>> > to the server).
>> >
>> > That is true, but does not mention the requestSentRetryEnabled flag
>> >
>> > Currently, DefaultHttpRequestRetryHandler Javadoc says:
>> >
>> > requestSentRetryEnabled true if it's OK to retry requests that have been sent
>> >
>> > Which is a bit misleading, as it only applies to non-idempotent
>> > requests - idempotent requests are always retried.
>> >
>> > So I think the current Javadoc could do with updating to avoid giving
>> > the impression that the flag can be used to prevent idempotent
>> > retries.
>> >
>> >> I am fine with changing the behavior, but in 4.4 branch.
>> >
>> > That would also need reflecting in the User Guide.
>> > And we would need to decide if retries should be separately enabled
>> > for idempotent requests.
>> >
>> > In the meantime, JMeter can create its own retry handler.
>>
>> Or so I thought ...
>>
>> Unfortunately it appears that the disconnect only occurs - or perhaps
>> is only detected - after the next request has been fully sent.
>>
>> This means that POST requests that fall foul of this won't be retried
>> anyway, and cannot be safely retried.
>>
>> What would perhaps work for JMeter would be a version of the stale
>> connection check that was conditionally applied to a connection.
>> For example only after the connection has been idle for a while.
>> Unfortunately unconditional stale checking affects performance too
>> much.
>>
>> The only other solution I can see at present is to be able to somehow
>> detect the disconnect earlier.
>
> Sadly, this just does not seem to work well with Java classic (blocking)
> I/O model. We, as a project, have been battling this problem for over 10
> years as far as I remember. Sometimes small requests can fully written
> into some kind of internal output buffer even though the underlying
> connection is no longer valid. I/O exception is thrown only on the
> subsequent read attempt.

I'd forgotten that - long time since I worked on raw TCP.

Did you ever experiment with disabling Nagle's algorithm?
Just curious.

>
>> For example after sending the first line of the request, would it be
>> possible to somehow check if the connection has been dropped?
>> And could this be done without compromising the performance?
>>
>
> It would hardly be any different than the stale check used presently.

In which case, is is possible to implement a conditional stale check?
e.g. every n requests or after a certain idle time?
That should reduce the overhead considerably.

BTW, could not find any docs on enabling/disabling the stale
connection check in the user guide.

> Oleg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Mime
View raw message