tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Filip Hanik - Dev Lists <devli...@hanik.com>
Subject Re: Tomcat bottleneck on InternalInputBuffer.parseRequestLine
Date Wed, 02 Jul 2008 21:37:25 GMT
Christophe Fondacci wrote:
> Thanks for your replies.
>
>
> Charles,
>
> I am logging long request in my client code, and a POST request like 
> http://S:8080/solr/select/ can take 250s with 12 posted NameValuePair 
> while this same request can pass OK in hundreds of ms. So I *think* 
> our requests are well formed (I log the request after the Solr server 
> response). According to you, what can cause tomcat to expect more data ?
> I like the "multithread problem on our client" suggestion and am going 
> to investigate in this way.
>
> Filip,
>
> I have many doubts about an "expected" behaviour.
> When our server behaves correctly, almost all http threads have the 
> following call stack :
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:485)
> at 
> org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:416)
> at 
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:442)
> at java.lang.Thread.run(Thread.java:619)
>
> They are waiting, not reading...
> My understanding of Tomcat (please correct me if i am wrong) is that 
> the Coyotte connector
> dispatches http requests to http threads when needed, though awaking 
> them from their waiting
> state. The http threads should only read one single request, process 
> and return, shouldn't they?
this is what happens,
1. thread goes into socketAccept, waiting for connection
2. thread reads request - parseRequestLine
3. thread executes the request - this is your servlet
4. thread goes into read request - parseRequestLine

if 4 times out (which it almost always does at some point, but using the 
connectionTimeout value, 20sec) then it closes the connection and goes 
back to the thread pool

since you most likely have a fair amount of timeouts happening, then you 
most likely will show lots of time shown in that method using a profiler.

set
maxKeepAliveRequests="1" for your connector, and see if you see a 
different behavior

Filip

>
> An http thread which is "reading" (executing parseRequestLine) is not 
> able to handle http request. Eventually they were already processing 
> some request. (again please correct me if i'm wrong).
>
> Can you give me more details about why you think that an http thread 
> which is reading a socket for hundreds of seconds is a normal behaviour?
>
> After a while, under heavy load, every http thread is doing 
> "parseRequestLine" and tomcat is not able to serve any more request 
> until these method calls finish. It can last 10 minutes before Tomcat 
> is able to serve new requests. When Tomcat starts to respond again, we 
> can clearly see that some http threads returned to their waiting state.
>
>
> Thank you.
> Christophe.
>
> ----- Original Message ----- From: "Filip Hanik - Dev Lists" 
> <devlists@hanik.com>
> To: "Tomcat Users List" <users@tomcat.apache.org>
> Sent: Wednesday, July 02, 2008 5:43 AM
> Subject: Re: Tomcat bottleneck on InternalInputBuffer.parseRequestLine
>
>
>> if you are using keep alive connections, (Tomcat default config)
>> then once a request is done, the system goes into reading the next 
>> request, and that is done by calling parseRequestLine, so this is not 
>> a bottleneck, but expected behavior.
>> if you switch to the NIO connector, that doesn't do block reading 
>> during a keep alive, your numbers would show differently. however, 
>> the perf numbers you are looking at are no cause for concern
>>
>> Filip
>>
>>
>> Caldarale, Charles R wrote:
>>>> From: Christophe Fondacci
>>>> [mailto:christophefondacci@nextep-softwares.com]
>>>> Subject: Tomcat bottleneck on InternalInputBuffer.parseRequestLine
>>>>
>>>> This problem may be related to the one listed here :
>>>> http://grokbase.com/profile/id:hNxqA0ZEdnD-6GYFRNs-iIkKEvF907F
>>>> NWdczKYQ719Q
>>>>
>>>
>>> I would have my doubts about a five-year-old problem description 
>>> being pertinent to today's Tomcat.
>>>
>>>
>>>> When the problem occurs, we can see threads which are
>>>> stucked with the following call stack :
>>>>
>>>> at java.net.SocketInputStream.socketRead0(Native Method)
>>>> at java.net.SocketInputStream.read(SocketInputStream.java:129)
>>>> at
>>>> org.apache.coyote.http11.InternalInputBuffer.fill
>>>> (InternalInputBuffer.java:700)
>>>> at
>>>> org.apache.coyote.http11.InternalInputBuffer.parseRequestLine
>>>> (InternalInputBuffer.java:366)
>>>>
>>>
>>> It looks like the parser is waiting for more of the request to show 
>>> up. Are you positive that the stalled requests are well-formed?  Is 
>>> it possible that there's something not quite thread safe in your 
>>> client on Server A that might be corrupting the request stream as 
>>> it's being generated?  Can you run a Wireshark or equivalent trace 
>>> and observe one of the slow ones?  (That may be tricky.)
>>>
>>>  - Chuck
>>>
>>>
>>> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE 
>>> PROPRIETARY MATERIAL and is thus for use only by the intended 
>>> recipient. If you received this in error, please contact the sender 
>>> and delete the e-mail and its attachments from all computers.
>>>
>>> ---------------------------------------------------------------------
>>> To start a new topic, e-mail: users@tomcat.apache.org
>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>
>>>
>>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To start a new topic, e-mail: users@tomcat.apache.org
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
>



---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message