httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Bawcombe" <bbawco...@promnetwork.com>
Subject RE: [users@httpd] Apache HTTPD Sporadic High CPU
Date Wed, 11 Apr 2012 17:57:40 GMT
We checked our APR version (1.4.5) and we should be OK for the bug you mention.

Here is the output from our httpd -V if this helps:

Server version: Apache/2.2.20 (Unix)
Server built:   Sep 30 2011 13:13:01
Server's Module Magic Number: 20051115:28
Server loaded:  APR 1.4.5, APR-Util 1.3.12
Compiled using: APR 1.4.5, APR-Util 1.3.12
Architecture:   32-bit
Server MPM:     Worker
threaded:     yes (fixed thread count)
    forked:     yes (variable process count)
Server compiled with....
-D APACHE_MPM_DIR="server/mpm/worker"
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_PROC_PTHREAD_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D DYNAMIC_MODULE_LIMIT=128
-D HTTPD_ROOT="/opt/apache/cdars_2.2.20"
-D SUEXEC_BIN="/opt/apache/cdars_2.2.20/bin/suexec"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"



-----Original Message-----
From: Jeff Trawick [mailto:trawick@gmail.com] 
Sent: Tuesday, April 10, 2012 9:24 PM
To: users@httpd.apache.org
Subject: Re: [users@httpd] Apache HTTPD Sporadic High CPU

On Tue, Apr 10, 2012 at 9:07 PM, Bill Bawcombe <bbawcombe@promnetwork.com> wrote:
> CA wasn't much help at all but that is pretty typical.
>
> We are requesting sudo access to pstack and truss and a few other utilities as my team
is application specific and we don't have root access.  I was hoping someone may have run
into an issue like this before but the one thread I've found on here and other places regarding
high CPU and ICX has gone unanswered since 2005.
>
> One solution we've researched is to reduce the number of threads spawned by apache. Our
thinking is that the number of threads generated is too large to be handled by the server.
 This is governed by the "ThreadsPerChild" parameter.  The current value is 25 and we were
thinking of reducing the value to 20.
>
> I should add that we are running in Worker MPM.

I doubt that the number of worker threads is the issue, and reducing from 25 to 20 isn't much
difference anyway.  (Another way to look at it is that you'll end up with more SiteMinder
threads as you increase the number of child processes ;) )

I hope your APR is >= 1.3.10.  (I think one of the symptoms from old bugs in Solaris event
port support was looping.)

>
> Bill
>
> -----Original Message-----
> From: Jeff Trawick [mailto:trawick@gmail.com]
> Sent: Tuesday, April 10, 2012 7:17 PM
> To: users@httpd.apache.org
> Subject: Re: [users@httpd] Apache HTTPD Sporadic High CPU
>
> On Tue, Apr 10, 2012 at 5:51 PM, Bill Bawcombe <bbawcombe@promnetwork.com> wrote:
>> We are sporadically experiencing high CPU with our Apache instances.
>> The CPU averages about 5% utilization normally and has run fine for 
>> over a year without an issue but over the past month we see spikes up 
>> to 100% utilization that never drop until the instance is bounced.  
>> We can go up to a week before we see the issue but we've also seen it 
>> happen multiple times in a day.  The logs are not indicating any 
>> errors for further investigation and we only have this issue in our Production environment.
>>
>>
>> Things we've tried:
>> -We've searched numerous topics on Apache HTTPD and high CPU and with 
>> regards to tuning parameters but considering we haven't had an 
>> increase in usage and our average CPU utilization is normally low we 
>> are reluctant to make tuning changes without understanding what else 
>> we may impact.
>>
>> -We ran prstat and the one anomaly we've seen is the Involuntary 
>> Context Switching (ICX) seems to be high when the CPU spikes.
>> Normally ICS is in the single digits but when the CPU spikes we see 
>> thousands.  From observation it appears that the ICX value and CPU 
>> utilization increase and drop simultaneously.
>>
>> Information regarding our environment:
>> - Apache HTTPD 2.2.20
>> - MOD_PROXY as a reverse proxy
>> - Solaris 10
>> - 2 servers for high availability and have experienced the issue on 
>> both servers
>>
>> One other note is that we are using CA's SiteMinder (authentication 
>> and
>> authorization) agent for Apache.  We've exhausted all of the support 
>> CA will give us regarding this issue and they have no other known 
>> issues from their customer base.
>
> What kind of information did CA gather?  SiteMinder has its own threads in the httpd
processes, and of course SiteMinder can run on the normal httpd threads.
>
> One obvious thing to try is to get a handful of pstacks of a high CPU process at 10 second
intervals when the problem occurs and see if you can find anything out of the ordinary there.
 You can also point truss to particular LWPs (as shown by pstack) for threads that look suspicious.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>



--
Born in Roswell... married an alien...

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Mime
View raw message