tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "K J.Sreekumar" <k.j.sreeku...@comprotechnologies.com>
Subject Re: Tomcat stops responding
Date Sat, 18 Dec 2010 12:27:10 GMT
We have done the tests you had mentioned.
here are our answers to your questions (i'm adding my name before my
comments so that its easier to identity who said what if this conversation
builds up further)-

*1.* What do you mean by : Browsers are unable to find application URLs
(which were previously processed normally).What error do you get in the
browser ? (Preferably, use a browser other than IE, or if you do use IE,
then in the options disable the "friendly error messages")
*
** Sreekumar.* Using Firefox 3.6x we get the following error
The connection was reset
The connection to the server was reset while the page was loading.
Additionally, the firebug net panel displays "Status - Aborted"

*2*. When the problem happens again : does the host still respond to "ping"?
*
**Sreekumar.* Yes, the host responds to ping. Additionally we have an Apache
server running and that continues to work normal. Other applications like
RDP and FTP are also NOT impacted.

*3.* On the server, in a command window, do : netstat -p tcp -anb
 and paste the complete result here
*
** Sreekumar.* NETSTAT was output as follows

Proto  Local Address          Foreign Address        State           PID
  TCP    0.0.0.0:21             0.0.0.0:0              LISTENING       1652
  [inetinfo.exe]

  TCP    0.0.0.0:80             0.0.0.0:0              LISTENING       1336
  [httpd.exe]

  TCP    0.0.0.0:135            0.0.0.0:0              LISTENING       704
  RpcSs
  [svchost.exe]

  TCP    0.0.0.0:445            0.0.0.0:0              LISTENING       4
  [System]

  TCP    0.0.0.0:1025           0.0.0.0:0              LISTENING       464
  [lsass.exe]

  TCP    0.0.0.0:1033           0.0.0.0:0              LISTENING       1420
  [erl.exe]

  TCP    0.0.0.0:1039           0.0.0.0:0              LISTENING       1880
  [erl.exe]

  TCP    0.0.0.0:1046           0.0.0.0:0              LISTENING       1568
  [cvd.exe]

  TCP    0.0.0.0:1047           0.0.0.0:0              LISTENING       1568
  [cvd.exe]

  TCP    0.0.0.0:1059           0.0.0.0:0              LISTENING       1568
  [cvd.exe]

  TCP    0.0.0.0:1066           0.0.0.0:0              LISTENING       1652
  [inetinfo.exe]

  TCP    0.0.0.0:1086           0.0.0.0:0              LISTENING       1416
  [EMCliSrv.exe]

  TCP    0.0.0.0:1105           0.0.0.0:0              LISTENING       2932
  [ManagementAgentNT.exe]

  TCP    0.0.0.0:1106           0.0.0.0:0              LISTENING       2932
  [ManagementAgentNT.exe]

  TCP    0.0.0.0:1311           0.0.0.0:0              LISTENING       2904
  [dsm_om_connsvc32.exe]

  TCP    0.0.0.0:3389           0.0.0.0:0              LISTENING       3844
  TermService
  [svchost.exe]

  TCP    0.0.0.0:4369           0.0.0.0:0              LISTENING       1440
  [epmd.exe]

  TCP    0.0.0.0:5222           0.0.0.0:0              LISTENING       1880
  [erl.exe]

from another workstation :
 - open a command window
 - enter : telnet (hostname) (port)     (the port on which Tomcat listens,
presumabyl 80)
   you will not get a prompt, you will get a blank screen.
 - now you have to type blind, because you will not get an echo :
   GET / HTTP/1.1
   <CR>
   <CR>
   (meaning 2 times "return"

 what do you see ?

- do the same from a command window on the server itself, using "localhost"
as the hostname.  Same result ?

***Sreekumar* - The telnet output was as follows (same from local machine
and a different workstation)

HTTP/1.1 400 Bad Request
Server: Apache-Coyote/1.1
Transfer-Encoding: chunked
Date: Sat, 18 Dec 2010 09:57:04 GMT
Connection: close

0

Connection to host lost.

Next we tried with /HTTP/*1.0 *instead - In this case, when on the local
machine/server, we did not get any response and the telnet exited
immediately after the 2nd [ENTER].

Also, we checked telnet /HTTP/*1.1* after restarting the machine (when
everything was running fine), it still gives a BAD REQUEST.  But telnet
requests to /HTTP/*1.0* started working.


***Sreekumar* - The performance counters (after tomcat stopped responding)
from the Window Perfmon for tomcat6.exe were as follows:

Elapsed time - 10666 (sec)
% processor time - 0.000
IO Data bytes per sec - 0.000
Handle count - 730
Page file bytes - 1252765696
Page file bytes Peak - 1293123584
Pool non paged bytes - 42576
Pool paged bytes - 76356
Priority base - 8
Private bytes - 1252765696
Thread count - 77
Virtual bytes - 1813188608
Virtual bytes Peak - 1818099712
Working set - 872779776
Working set Peak – 900476928

ALL OTHER COUNTERS WERE “0”

***Sreekumar *- Following are relevant apache error logs. Note, apache is
running on port 80 and forwarding requests (using mod_proxy) to tomcat on
port 8080.

[Sat Dec 18 04:02:54 2010] [error] [client 125.18.244.200] (OS 10054)An
existing connection was forcibly closed by the remote host.  : proxy: error
reading status line from remote server localhost

[Sat Dec 18 04:02:54 2010] [error] [client 125.18.244.200] proxy: Error
reading from remote server returned by /rel/


***Sreekumar *- I'm also attaching the complete thread dump output as
"threadDump.txt"

Please let me know what could be hinted by these results and if any further
tests are required to resolve this issue.

Thanks



On Fri, Dec 17, 2010 at 7:29 PM, André Warnier <aw@ice-sa.com> wrote:

> K J.Sreekumar wrote:
>
>> Hello All,
>>
>> We are hitting on a weird problem with tomcat 6 production server (which
>> is
>> a serious concern for us now). Hoping to get help from the you guys
>>
>> This is the problem -
>>
>>   1. Our tomcat 6 server (version 6.0.29) does not respond after working
>>
>>   for a couple of days. Browsers are unable to find application URLs
>> (which
>>   were previously processed normally).
>>   2. The logs does not show any error.
>>   3. There are only a total of 50-60 threads in the thread dump, mostly in
>>
>>   WAIT state. Also, it does not show any BLOCKING or LOCKED threads.
>>   4. Memory utilization was 400MB only
>>   5. TOMCAT restart does not fix the problem. The machine needs to be
>>
>>   restarted to resume normal operation.
>>   6. Netstat did not reveal any extraneous connections
>>
>>
>> We are running Tomcat 6.0.29 on the following platform -
>>
>> OS: Windows server 2003 R2 Enterprise Edition SP2
>> Hardware: Intel Xeon L5520 2.27 GHz (16 CPU threads )
>>
>>  The fact that restarting Tomcat does not help, and that there is no error
> in the logs, raises a strong suspicion that the problem may be external to
> Tomcat.
> So, just to eliminate certain side-issues :
>
> 1) what do you mean by :
> >    1. ... Browsers are unable to find application URLs (which
> >    were previously processed normally).
>
> What error do you get in the browser ? (Preferably, use a browser other
> than IE, or if you do use IE, then in the options disable the "friendly
> error messages")
>
>
> 2) When the problem happens again :
>
> - does the host still respond to "ping" ?
>
> - on the server, in a command window, do : netstat -p tcp -anb
>  and paste the complete result here
>
> - from another workstation :
>  - open a command window
>  - enter : telnet (hostname) (port)     (the port on which Tomcat listens,
> presumabyl 80)
>    you will not get a prompt, you will get a blank screen.
>  - now you have to type blind, because you will not get an echo :
>    GET / HTTP/1.1
>    <CR>
>    <CR>
>    (meaning 2 times "return"
>
>  what do you see ?
>
> - do the same from a command window on the server itself, using "localhost"
> as the hostname.  Same result ?
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>

Mime
View raw message