Mailing-List: contact users-help@tomcat.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Tomcat Users List" <users@tomcat.apache.org>
Received-SPF: pass (athena.apache.org: domain of aw@ice-sa.com designates
 212.85.38.229 as permitted sender)
Message-ID: <52BCC9C1.1020108@ice-sa.com>
Date: Fri, 27 Dec 2013 01:28:49 +0100
From: =?windows-1252?Q?Andr=E9_Warnier?= <aw@ice-sa.com>
Reply-To: Tomcat Users List <users@tomcat.apache.org>
User-Agent: Thunderbird 2.0.0.24 (Windows/20100228)
MIME-Version: 1.0
To: Tomcat Users List <users@tomcat.apache.org>
Subject: Re: Request Timeout and empty post data issue
References: 
 <9B195074700C78448AE342FD2B3E427C2BB0A9FD@MBX029-W1-CA-2.exch029.domain.local>
 <9B195074700C78448AE342FD2B3E427C2BB0AA3B@MBX029-W1-CA-2.exch029.domain.local>,<52BCB971.9040800@apache.org>
 <9B195074700C78448AE342FD2B3E427C2BB0AB1E@MBX029-W1-CA-2.exch029.domain.local>
In-Reply-To: 
 <9B195074700C78448AE342FD2B3E427C2BB0AB1E@MBX029-W1-CA-2.exch029.domain.local>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 8bit

Peter Rifel wrote:
>> ________________________________________
>> From: Mark Thomas [markt@apache.org]
>> Sent: Thursday, December 26, 2013 3:19 PM
>> To: Tomcat Users List
>> Subject: Re: Request Timeout and empty post data issue
>>
>> On 26/12/2013 20:45, Peter Rifel wrote:
>>> Hello,
>>>
>>> I'm currently running Tomcat 7.0.42 on Ubuntu 12.04 with OpenJDK
>>> 1.7.0_25.  I'm using Apache Tomcat Native library 1.1.27 with APR
>>> version 1.4.6.
>>>
>>> I'm noticing in my access logs that some of our POST requests don't
>>> have any POST data and all have response times of a few ms over
>>> 20000ms.  I'm trying to figure out whether this issue is client side
>>> or server side.  The response code and response size for these
>>> requests are normal.  Can anyone tell me under what circumstances
>>> this would happen?
>> It depends.
>>
>> The connectionTimeout is also the socket timeout. If at any time Tomcat
>> is (trying to) read data from the client or write data to the client and
>> no data is available (reading) or read (writing) for the timeout period
>> then the read / write will fail.
>>
>> It sounds like the client sends the request and then either doesn't send
>> the whole body, only sends part of the body or pauses for longer than
>> connectionTimeout while sending the body. The request body is therefore
>> missing / incomplete so can't be processed. Whether that results in an
>> error response will depend on your application.
>>
>>>  I noticed that our connector's connectionTimeout
>>> is set to 20000ms, but it wouldn't make sense for that value to be a
>>> part of this issue because a connection timeout only occurs when the
>>> URI hasn't been received by tomcat in that amount of time, which is
>>> clearly not happening here (I was able to confirm this with telnet; a
>>> connection timeout will not write anything to the access logs).
>> The above statement is incorrect. See the explanation preceding it.
> 
> 
> This is what the documentation on the Configuration Reference page says, so if it is incorrect it should be updated.
> 
> 
>>> The "request" is making it to my servlet (logging confirms this) but
>>> for some reason tomcat doesn't see any request parameters and all of
>>> the response times in our access logs are just above 20 seconds.
>>> Does this mean that my servlet is taking 20 seconds to process the
>>> request?
>> Unlikely. It almost certainly means Tomcat is waiting 20s for the
>> request body to turn up.
>>
>>
>>> Is there some other timeout value somewhere that defaults
>>> to 20 seconds?
>> No.
>>
>>> Is there a way for me to see exactly what is taking
>>> so long?
>> Wireshark.
>>
>>> Its incredibly hard to debug this because we cant reproduce
>>> this bug on our own without any post data and <1% of our production
>>> traffic is having this issue.  To try and gather more info on this, I
>>> added a servlet filter that logs all of our POST request parameters
>>> to the access log and I can confirm that there are no parameters on
>>> these specific requests.
>> That makes it look very much like a client issue. I'd log the size of
>> the request body as well. A request body size of > 0 and no POST data
>> would point to a client issue.
>>
>>> The lack of post data makes me think its a client issue, but that
>>> doesn't explain why all of these requests take ~20 seconds to be
>>> processed.
>> See above.
>>
>> Mark
>>
> 
> 
> I�m logging the Content-length header and it has expected values, mostly between 300 and 800 bytes.  The requests are coming from a custom flash player that is being in embedded in users� browsers that are out of our control, so it does make it difficult to debug with only 1% of our traffic seeing this problem.  I will run wireshark on tomcat to try and get a better idea of where the issue is, thank you for the suggestion.
> 

Suggestion: if the custom player is under your control, make sure that the problematic 
requests are indeed coming from this player, and not from some (malicious or not) other 
source.  Perhaps some additional "X-" header in the request, with some time-variable key 
that can be checked on the server side ?
You probably don't want to spend a lot of time inspecting thousands of packets if the 
reason is a misbehaving robot or monitoring agent or similar.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org