httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "RAY, DAVID" <DAVID....@tccd.edu>
Subject [users@httpd] RE: AIX OS Patch Breaks Apache/Tomcat
Date Wed, 01 Jul 2015 20:00:18 GMT
>-----Original Message-----
>From: RAY, DAVID [mailto:DAVID.RAY@tccd.edu]
>Sent: Tuesday, June 30, 2015 12:10 PM
>To: users@tomcat.apache.org
>Subject: AIX OS Patch Breaks Apache/Tomcat
>
>Hello.
>
>I am running Apache 2.2.29 and Tomcat 7.0.59 with tomcat connector(mod_jk) version 1.2.40
on AIX version 7.1 server.  Started having problems this morning after AIX OS was patched
to AIX 7.1 TL 03 SP 04  and openssh to >6.0.0.6108.  WebAdvisor runs fine immediately after
apache is started or restarted.  However its response slowing down.  AIX server CPU steadily
increases and approaches 100% after running for 5 or 10 minutes under heavy >load..  Seeing
http process accumulate.  Not seeing much traffic at all in Tomcat server status.  Recompiled
apache, tomcat connector, and Tomcat native.  Still no luck.  Later determined there is a
bug with openssh 6.0.0.6.108 >and tried 6.0.0.6109.  Recompiled again.  Still no improvement.
 Just booted from Clone backup taken before the AIX patch and everything  runs fine, no problems.
 Anyone else have this experience with AIX >patches/updates?  Not really much to see in
error logs.  Tomcat is configured to run with java6 32 bit:
>
>IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 AIX ppc-32 jvmap3260sr16-20141216_227499 (JIT
enabled, AOT enabled)
>J9VM - 20141216_227499
>JIT  - r9_20140523_64469ifx3
>GC   - GA24_Java6_SR16_20141216_1020_B227499)
>JCL  - 20141216_01
>
>
>Thank you.
>David


We have determined that that this problem is triggered by Apache server-status.   I have always
used server-status to monitor our production system, no problem. I rarely use server-status
to monitor our test system because there is nothing to see there due to extremely light traffic.
 But after the AIX patch, server-status gradually causes the sever CPU to be maxed out.  This
problem was just duplicated on our test server.  The CPU drops back to normal for that server
when server-status is stopped.  Example of server-status URL I use in my IE 11 browser:  https://website.com/server-status?refresh=5

Has anyone else had this experience?  I guess for now, I will just have to stop using server-status
to monitor Apache so that we can patch our AIX system.

Interesting how server-status is actually showing the gradual CPU  increase.  When this snapshot
was taken, server-status had just moved from line 9 to line 10, where the W is.   The 53.55
at line 9 is what the CPU usage was when this snapshot was taken:

0-0 17039448 0/13/13 _  48.65 52 371 0.0 0.09 0.09  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/13/13 _  49.14 47 305 0.0 0.10 0.10  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/13/13 _  49.64 41 297 0.0 0.11 0.11  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/13/13 _  50.46 35 554 0.0 0.11 0.11  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/13/13 _  51.58 23 380 0.0 0.12 0.12  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/13/13 _  51.04 29 373 0.0 0.09 0.09  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/13/13 _  52.19 17 399 0.0 0.14 0.14  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/13/13 _  52.93 11 528 0.0 0.11 0.11  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/13/13 _  53.55 5 438 0.0 0.11 0.11  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/12/12 W  40.99 0 0 0.0 0.09 0.09  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/12/12 _  43.98 104 256 0.0 0.09 0.09  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/12/12 _  41.50 133 355 0.0 0.09 0.09  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/12/12 _  42.01 127 338 0.0 0.10 0.10  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/11/11 _  42.47 122 237 0.0 0.11 0.11  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/12/12 _  42.93 116 325 0.0 0.10 0.10  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/12/12 _  43.45 110 320 0.0 0.11 0.11  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/12/12 _  44.41 99 264 0.0 0.11 0.11  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/12/12 _  44.91 93 368 0.0 0.12 0.12  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/12/12 _  46.99 70 378 0.0 0.08 0.08  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/12/12 _  45.48 87 365 0.0 0.11 0.11  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/12/12 _  45.91 82 256 0.0 0.10 0.10  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/12/12 _  48.06 58 254 0.0 0.09 0.09  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/12/12 _  46.41 76 294 0.0 0.09 0.09  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1
0-0 17039448 0/12/12 _  47.53 64 368 0.0 0.09 0.09  xxx.xxx.xx.x website.com GET /server-status?refresh=5
HTTP/1.1


David

Mime
View raw message