httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stevie <iglabl...@yahoo.com>
Subject [users@httpd] Re: Assistance using server-status to determine if configuration could be causing issues or if it may be system compromise
Date Mon, 07 May 2012 19:17:21 GMT
Is no one able to provide any feedback or help with this? Have I worded my question in a bad
way? I'll word it differently if it helps. Can someone speak to the longevity of the child
process states I mentioned below? 


----- Original Message -----
From: Stevie <iglablues@yahoo.com>
To: "users@httpd.apache.org" <users@httpd.apache.org>
Cc: 
Sent: Thursday, April 26, 2012 1:07 PM
Subject: Assistance using server-status to determine if configuration could be causing issues
or if it may be system compromise

I have Apache 2.2.14 running on Ubuntu 10.04. The server has been running for almost a year
without incident. The server is a front-end web server which passes requests for dynamic content
(Java) to Tomcat on other servers using mod_jk. April 24th the web server started responding
slugglishly to client requests (as reported by clients) before ultimately becoming unresponsive.
It had to be hard rebooted to bring it back up. I found no evidence of what caused the crash
after the server came back up, but I had noticed that there were a lot of Apache processes
running (~146) right before everything crapped the bed. The MaxClients directive is set to
the default of 150 and this has been sufficient all this time. I changed it to 200 and immediately
saw that there was a jump in child process numbers almost immediately, like it couldn't wait
to launch more child processes. I didn't see anything in the error log about going over the
MaxClient limit so I'm not
sure if I'm chasing squirrels here. In a nutshell, what I'm looking for is help is figuring
out if MaxClients/KeepAlives are potential performance problems in my config, or whether I
might be looking at some outside influence (compromised system). 

The server is running mod_php so it's using the prefork MPM. Here are the relevant lines from
apache2.conf:

<IfModule mpm_prefork_module>
    StartServers          5
    MinSpareServers       5
    MaxSpareServers      10
    MaxClients          200
    MaxRequestsPerChild   0
</IfModule>


KeepAlive On
MaxKeepAliveRequests 100

KeepAliveTimeout 15

I enabled the server-status page and briefly turned ExtendedStatus On (didn't want to leave
it on if it could decrease performance). Here's what I saw:

KWKKKKKKKKKKKKKKKKKKKKKKKKKCKKKKKKWKCKKKKKKKKCKKKWSSSSS.........
.....K...................K.....W.......K........................
..K.....K....K..........W...G...KK...................W.......... 
.C.....K........................................................

I had a lot of KeepAlives that I recognized, like this:
3-3 886 6/6/6 K 0.01 3 1 7.1 0.01 0.01 24.227.88.162 clientname.com POST /RequestHandler HTTP/1.1

This makes sense to me, and the question I have regarding this is whether I should decrease
the KeepAlive Timeout number since it seems like that is part of what is causing Apache to
spawn so many child processes. 

The other bit is that along with the above lines in server-status I am seeing things like:

136-2 747 0/0/0 K 0.00 1335452011 0 0.0 0.00 0.00    
141-2 752 0/0/0 K 0.00 1335452011 0 0.0 0.00 0.00    
152-2 764 0/0/0 W 0.00 1335452011 0 0.0 0.00 0.00    
156-2 768 0/0/0 G 0.00 1335452011 0 0.0 0.00 0.00    
160-2 772 0/0/0 K 0.00 1335452011 0 0.0 0.00 0.00    
161-2 773 0/0/0 K 0.00 1335452011 0 0.0 0.00 0.00    
181-2 793 0/0/0 W 0.00 1335452011 0 0.0 0.00 0.00    
193-2 805 0/0/0 C 0.00 1335452011 0 0.0 0.00 0.00    
199-2 811 0/0/0 K 0.00 1335452011 0 0.0 0.00 0.00 
If I understand the scoreboard key correctly, this means that there are child processes that
have been in either a KeepAlive, closing, or replying state that have no connection on the
other end. It's essentially like they're stuck or hanging. Is this an accurate understanding?
I also converted the seconds in the SS column to get an idea of how long they've been that
way, and it comes out to something like 15,000 days! That's not right obviously. Am I doing
something wrong in that calculation, or misunderstanding what it really means? 

Thanks in advance,
SC


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Mime
View raw message