jakarta-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sebb <seb...@gmail.com>
Subject Re: Lots of CLOSE_WAIT with HC4 HTTP implementation
Date Wed, 21 Sep 2011 22:42:15 GMT
On 21 September 2011 22:29, Milamber <milamber@apache.org> wrote:
> Hello,
>
> Sorry, but the problem don't seems resolve.
>
> Today, I try a load test on a Content Delivery Network (CDN) with HC3.1
> vs HC4 with last JMeter trunk (r1173449).
>
> My script :
> Test only static resources (gif, png, jpeg) with direct HTTP Request
> 100 users
> Ramp up : 100 secs
> Duration test : 300 secs (5min)
> Each iteration execute 39 HTTP requests
> HTTP requests : Keep Alive true
>
>
> I have 2 issues (on 8 same tests: 4 HC3.1 and 4 HC4.1) :
>
> *_Issue 1_*
> With HC 3.1 : always no error
> With HC 4.1 : always some errors at the beginning of load test
>
> Stack trace (on view result tree: response data) is :
> org.apache.http.NoHttpResponseException: The target server failed to respond
>    at
> org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:101)
>    at
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
>    at
> org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:281)
>    at
> org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
>    at
> org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:219)
>    at
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
>    at
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
>    at
> org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:645)
>    at
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464)
>    at
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
>    at
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
>    at
> org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(HTTPHC4Impl.java:265)
>    at
> org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sample(HTTPSamplerProxy.java:62)
>    at
> org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1010)
>    at
> org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:996)
>    at
> org.apache.jmeter.threads.JMeterThread.process_sampler(JMeterThread.java:383)
>    at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:276)
>    at java.lang.Thread.run(Thread.java:722)
>
>
> HC 4 Summariser (begin) :
> summary +    55 in   2,2s =   24,9/s Avg:    71 Min:    15 Max:  1677
> Err:     0 (0,00%)
> summary +  5027 in  30,0s =  167,5/s Avg:   103 Min:     1 Max:  1816
> Err:    14 (0,28%)
> summary =  5082 in  32,2s =  157,8/s Avg:   103 Min:     1 Max:  1816
> Err:    14 (0,28%)
> summary +  5013 in  31,4s =  159,7/s Avg:   283 Min:     2 Max:  3500
> Err:    11 (0,22%)
> summary = 10095 in  62,2s =  162,3/s Avg:   192 Min:     1 Max:  3500
> Err:    25 (0,25%)
> summary +  5075 in  31,6s =  160,7/s Avg:   449 Min:    37 Max:  7999
> Err:     0 (0,00%)
> summary = 15170 in  92,2s =  164,5/s Avg:   278 Min:     1 Max:  7999
> Err:    25 (0,16%)
> summary +  4984 in  34,8s =  143,2/s Avg:   587 Min:    40 Max: 11028
> Err:     0 (0,00%)
> summary = 20154 in 122,2s =  164,9/s Avg:   355 Min:     1 Max: 11028
> Err:    25 (0,12%)
> */_
> Issue 2 :_/*
>
> With HC3.1 after ramp up:
>  netstat -tplna|grep java|grep ESTA|wc -l
> 100
>
> With HC4.1 at beginning and after ramp up :
> netstat -tplna| grep java | grep ESTA | wc -l
> 1145 (value change between ~1000-1150)
>
> I make a tcp dump and analyze with wireshark.
> With HC3.1 : 0-42 secs : 43 TCP conversations (inside several HTTP GET
> and response)
> With HC4.1 : 0-44 secs : 3011 TCP conversations (inside one HTTP GET and
> one response)

Do you mean that each HC4.1 conversation only contains a single
request/response?

Can you enable debug logging for HTTPHC4Impl ?
This should show if the clients are being reused or not.

I suggest you reduce thread and loop counts, or the log file will be very big.

[I think we need to fix this before releasing 2.5.1]

> HTTP conversations show Keep-Alive header (HC3 and HC4)
>
> I suppose that is a close keep alive connection problem with JMeter.
> Perhaps 2 issues are linked?

Dunno yet.

> Milamber
>
>
> Le 24/08/2011 11:40, sebb a ecrit :
>> On 24 August 2011 02:43, sebb <sebbaz@gmail.com> wrote:
>>
>>> Looks like HTTPHC4Impl is not reusing HttpClient instances.
>>>
>>> It should only create one per thread per unique URL + some other fields.
>>>
>>> It seems not to be finding the previously created instances, so it
>>> creates a new one for every sample.
>>> Not sure why yet; probably something simple.
>>>
>> Very simple.
>>
>> The equals() method was returning false if the other object was the
>> correct type, instead of returning false for the incorrect type.
>> So the hash keys never matched.
>>
>> Nightly builds from r1160957 will have the fix.
>>
>>
>>> However the code does at least seem to be closing the connections at
>>> end of thread.
>>>
>>> On 23 August 2011 17:13, Milamber <milamber@apache.org> wrote:
>>>
>>>> [snip]
>>>>
>>>>>> I believe is a HTTPClient issue, on close connection?
>>>>>> (http://www.sunmanagers.org/pipermail/summaries/2006-January/007068.html)
>>>>>>
>>>>>>
>>>>> I think it's more likely to be that JMeter is not tidying up the HC4
>>>>> connections properly.
>>>>>
>>>>> Are you using Keep-Alive?
>>>>>
>>>>>
>>>> Yes.
>>>>
>>>>
>>>>> Can you reproduce the problem with a simpler test case, e.g. one that
>>>>> could be run against a local server.
>>>>>
>>>>>
>>>> Yes on my laptop (Debian 6 / 64 bits / openjdk 1.6u18)
>>>>
>>>> in attachment JMeter testcase.
>>>>
>>>> With HC4 impl:
>>>> summary =  2590 in  72.4s =   35.8/s Avg:   994 Min:   487 Max:  4806
>>>> Err:     0 (0.00%)
>>>>
>>>> Stats at ~ 72secs :
>>>> netstat -tuplna|grep java |wc -l;
>>>> 1494
>>>> netstat -tuplna|grep java|grep ESTA|wc -l;
>>>> 167
>>>> netstat -tplnau|grep java|grep CLOSE_WAIT|wc -l;
>>>> 1337
>>>>
>>>>
>>>> With HC3 impl:
>>>> summary =  4498 in  83.2s =   54.1/s Avg:   448 Min:   246 Max:  2489
>>>> Err:     0 (0.00%)
>>>>
>>>> Stats at ~83 secs:
>>>> netstat -tuplna|grep java |wc -l;
>>>> 61
>>>> netstat -tuplna|grep java|grep ESTA|wc -l;
>>>> 60
>>>> netstat -tplnau|grep java|grep CLOSE_WAIT|wc -l;
>>>> 0
>>>>
>>>> NB: more requests in same time with HC3.
>>>> NB2: HC3 ESTABLISHED connections is lesser than HC4 (60 users in JMeter
>>>> scripts)
>>>>
>>>>
>>>> Milamber
>>>>
>>>>
>>>
>>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: dev-help@jakarta.apache.org


Mime
View raw message