hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pete Keyes <PKe...@starbucks.com>
Subject Re: NoHttpResponseException received at startup
Date Fri, 06 May 2016 14:13:44 GMT
Thanks Oleg.  

The keyword “reset” is enough to help us point networking in a plausible
direction.
  - since this is a symptom associated with a broker connection this
confirms why each retry always fails
    within the same 5ms elapsed time;
  - we currently allow a retry using the same execution context;
      - obviously a fruitless waste of resource time;

— 
Pete






On 5/6/16, 5:24 AM, "Oleg Kalnichevski" <olegk@apache.org> wrote:

>On Thu, 2016-05-05 at 14:46 +0000, Pete Keyes wrote:
>> Update for the elapsed time for these calls that fail strictly at
>>startup.
>>  The stair-step 5/25/500... was strictly due to a pause we insert
>>between
>> retries.  All 4 tries fail within 5ms.
>> 
>> Given the configuration settings for CloseableHttpClient a failure
>>within
>> 5ms seems unlikely.  The trip to the F5 VIP and back would barely
>>complete
>> within 5ms.
>> 
>> 
>> Any thoughts on how to stabilize this a bit at startup?  Or glean more
>> information about why apache-hc would fail with NoHttpResponseException
>>so
>> quickly.  
>
>The best course of action would be to capture IP packets with a traffic
>analyzer such as Wireshark and to try to figure out why TCP connections
>get reset.
>
>Oleg 
>
>
>
>> Stack trace for the error:
>>     org.apache.http.NoHttpResponseException:
>> crm-asp-prod.starbucks.net:443 failed to respond
>>     
>> 
>>org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttp
>>Re
>> sponseParser.java:143)
>>     
>> 
>>org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttp
>>Re
>> sponseParser.java:57)
>>     
>> 
>>org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser
>>.j
>> ava:261)
>>     
>> 
>>org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(D
>>ef
>> aultBHttpClientConnection.java:165)
>>     
>> 
>>org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.jav
>>a:
>> 167)
>>     
>> 
>>org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpReques
>>tE
>> xecutor.java:272)
>>     
>> 
>>org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.
>>ja
>> va:124)
>>     
>> 
>>org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java
>>:2
>> 71)
>>     
>> 
>>org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184
>>)
>>     org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
>>     
>> 
>>org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110
>>)
>>     
>> 
>>org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClie
>>nt
>> .java:184)
>>     
>> 
>>org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClie
>>nt
>> .java:82)
>> 
>> 
>> 
>> ‹ 
>> Pete Keyes
>> 
>> 
>> 
>> 
>> 
>> 
>> On 5/4/16, 11:13 AM, "Pete Keyes" <PKeyes@starbucks.com> wrote:
>> 
>> >We have an odd issue wherein at Tomcat(TomEE) startup apache-hc
>>execution
>> >requests throw a NoHttpResponseException within less than 1 second.
>> >After a few minutes this behavior completely stops and isn¹t seen again
>> >until the next time we stop and restart Tomcat(TomEE).
>> >
>> >Basic apache-hc configuration information:
>> >  - single static CloseableHttpClient instance initialized at container
>> >startup;
>> >  - all requests managed via PoolingHttpClientConnectionManager
>> >  - connection timeout 6000ms
>> >  - read timeout 6000ms
>> >  - we use 4 retries
>> >  - the typical timing for each
>> >HttpClient.execute(HttpUriRequest,HttpContext) during this startup
>>period
>> >is that they fail very fast:
>> >      try-1: ~  5ms
>> >      try-2: ~250ms
>> >      try-3: ~500ms
>> >      try-5: ~700ms
>> >We measure the elapsed milli-seconds with respect to ³execute()².  It
>> >seems very odd that execute() fails so fast given the apache-hc
>>settings.
>> > It isn¹t every execution request that fails.  But, a very high
>> >percentage of them do exhibit this behavior during the initial ~5
>>minutes
>> >after Tomcat(TomEE) begins dispatching inbound requests.
>> >
>> >The inbound HTTP request volume inbound is ~70 RPS(requests per
>>second).
>> >
>> >After ~5 minutes this type of error never repeats.  We have logging in
>> >place that verifies all the apache-hc components are fully instantiated
>> >prior to any request being processed.
>> >
>> >Version information:
>> >  apache-hc: 4.5.2
>> >  java     : 1.8r60
>> >  tomcat   : 7.0.68
>> >  tomee    : 1.7.4
>> >
>> >
>> >Any ideas at all given the minimal information that I have to provide
>>are
>> >greatly appreciated.
>> >
>> >‹
>> >Pete
>> >
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>> 
>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>For additional commands, e-mail: httpclient-users-help@hc.apache.org
>

Mime
View raw message