Mailing-List: contact users-help@httpd.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@httpd.apache.org
Received-SPF: pass (athena.apache.org: local policy includes SPF record at
 spf.trusted-forwarder.org)
Message-ID: <533D6EE5.1070705@christopherschultz.net>
Date: Thu, 03 Apr 2014 10:23:33 -0400
From: Christopher Schultz <chris@christopherschultz.net>
Reply-To: users@httpd.apache.org
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9;
 rv:24.0) Gecko/20100101 Thunderbird/24.4.0
MIME-Version: 1.0
To: users@httpd.apache.org
References: <533D6A6C.9000908@christopherschultz.net>
In-Reply-To: <533D6A6C.9000908@christopherschultz.net>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="6u6DDtvHd5s0K34aT7ceId6qEW0rXliBG"
Subject: Re: [users@httpd] High CPU Usage in Amazon EC2

--6u6DDtvHd5s0K34aT7ceId6qEW0rXliBG
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

All,

I forgot to mention that most of our traffic is over SSL. OpenSSL
version is OpenSSL 1.0.1e-fips 11 Feb 2013.

Thanks,
-chris

On 4/3/14, 10:04 AM, Christopher Schultz wrote:
> All,
>=20
> I'm having a problem in production I've never seem before. We are
> running a pair of AWS EC2 m1.micro web servers where only one of them i=
n
> really in service at any given time. The httpd instance serves some
> static content and forwards a great deal of traffic via stunnel to a
> single back-end Tomcat server using mod_jk 1.2.37. We have been running=

> under this configuration for several years with no problems.
>=20
> Last weekend, we upgraded our OS to Amazon Linux 2014.03 (32-bit) from
> Amazon's previous version (I can't remember which one), including the
> package-refresh that comes with it for httpd. The current kernel versio=
n
> is 3.10.34. The current httpd version is 2.2.26. The package name is
> "httpd-2.2.26-1.1.amzn1.i686" if anyone is interested. We are using a
> prefork MPM with the following (default) settings:
>=20
> StartServers       8
> MinSpareServers    5
> MaxSpareServers   20
> ServerLimit      256
> MaxClients       256
> MaxRequestsPerChild  4000
>=20
> What I can observe is that the CPU load average is rising from the usua=
l
> sub-2.0 value to sometimes as high as 70. That's seventy, not
> seven-point-oh.
>=20
> I see no errors in the log, and httpd doesn't seem to be dropping any
> requests... just running very very slowly.
>=20
> It seems to come in waves: the load will go up, and everything will slo=
w
> down, and then we'll get a reprieve.
>=20
> I can see 22 server processes running right this moment, but the load
> average has dropped back to 0.05.
>=20
> I've enabled ExtendedStatus and it really doesn't look like there is a
> huge number of requests being served. Less than 1 req/sec. This is *not=
*
> a high-load server. I can see some of the httpd child processes using
> 20% or more of the CPU as reported by 'top'.
>=20
> Is there a good way for me to determine what those processes are doing?=

> As this is a modestly-used server, I can probably enable additional
> logging without too much trouble.
>=20
> Any help anyone can provide would be very much appreciated.
>=20
> Thanks,
> -chris
>=20


--6u6DDtvHd5s0K34aT7ceId6qEW0rXliBG
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJTPW7lAAoJEBzwKT+lPKRYQmIQAKun6P51QgpNse6QcqrknzDg
0DsnQeM7YKUa9uDlpYJMabGeElFZE7y8XkCOLBqXoOX7vKwypNlM0cSz399rchWs
0HBMJQWJjRPWeoCCG/osY3R/7H0PAfryXmQtIqRqe/1VvcnOtmDnbV4qZLn5gVwP
y0t16kthtd93x/MzGp71zSvmLn7h/bvTKzfMHJBT5TEBFgW/Phil5LJEUjPLLHN6
CM5ZQb7RvdXK2uDActWIRPQFAYKlpiTQI0CWm7HKQ++BRMVDL9F5h1V7UBADGp/A
Hlxx6jPMwU0R2H66gateQfYQawhWbxJNktnXZJ3We+20KzLh2SNup79IYrQQWD6x
8rZ3bV9enWMUcOXdmg4UJ2TlhYNY2lui6IKFVHO+E1Td9VCk6x3YUe7wjqyTCT9C
W462EetnUTQUiSPoXtRi7Xvos4XZKdGE5apyXqqMlusxX1Od7ZOe8hUQAVfpK5id
/NR4Xo/XD4CsXtrk28LnIWEk8u4d8eTQdmvgv+dDwFnpJei8QfwcFQDM/jlJ7Uzp
u6Df79dUWKXuSkZvf/wO2HLAUvUmy+JNbdmAC8Dz/eiBMWpiLTTP31ytn9iMfW/P
H0tBW1EU5DZPnp4MZYGhdHFwpWPyA6zDa1quv14y2pZMgzKIPOCpVVu7LjEofDB8
5tU+qiEsbtlPcroUkEa5
=OKER
-----END PGP SIGNATURE-----

--6u6DDtvHd5s0K34aT7ceId6qEW0rXliBG--