www-apache-bugdb mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Schmidt <da...@galaxy-net.net>
Subject os-solaris/1733: performance on 50+ web servers drops off after 3-5 days.
Date Mon, 26 Jan 1998 20:38:54 GMT

>Number:         1733
>Category:       os-solaris
>Synopsis:       performance on 50+ web servers drops off after 3-5 days.
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    apache
>State:          open
>Class:          sw-bug
>Submitter-Id:   apache
>Arrival-Date:   Mon Jan 26 12:40:00 PST 1998
>Last-Modified:
>Originator:     david@galaxy-net.net
>Organization:
apache
>Release:        any
>Environment:
SunOS 5.5.1 Generic i86pc i386 i86pc runing apache 1.2 
128mgs memory
pentium 200mx

>Description:
System runs for "X" days eventually eating up all memory ("X" depends on amount of traffic)

Killing the process and restarting solves the problem for a few hours.

I have found this to be the same on all versions of apache.

memory does not seem to be returned to the os after httpd dies.

this causes performance to drom off over a few day peroid. eventually
the web server will become unstable and die. 

My only course of action is to reboot. What do you suggest?

here are ps and vmstat outputs:

/usr/ucb/ps -aux
USER       PID %CPU %MEM   SZ  RSS TT       S    START  TIME COMMAND
root     29511  4.2  0.7  992  828 pts/0    O 12:30:28  0:00 /usr/ucb/ps -aux
nobody   29409  0.6  1.1 2112 1328 ?        S 12:29:36  0:00 /usr/local/etc/htt
nobody   29448  0.6  1.0 2060 1276 ?        S 12:30:02  0:00 /usr/local/etc/htt
nobody   29411  0.6  1.0 2060 1276 ?        S 12:29:37  0:00 /usr/local/etc/htt
nobody   29418  0.6  1.1 2100 1316 ?        S 12:29:41  0:00 /usr/local/etc/htt
nobody   29412  0.5  1.1 2076 1292 ?        S 12:29:37  0:00 /usr/local/etc/htt
nobody   29338  0.5  1.1 2084 1300 ?        S 12:28:09  0:00 /usr/local/etc/htt
nobody   29443  0.5  1.1 2080 1296 ?        S 12:30:00  0:00 /usr/local/etc/htt
nobody   29420  0.5  1.0 2072 1288 ?        S 12:29:41  0:00 /usr/local/etc/htt
nobody   29419  0.5  1.0 2072 1288 ?        S 12:29:41  0:00 /usr/local/etc/htt
nobody   29406  0.5  1.1 2080 1296 ?        S 12:29:31  0:00 /usr/local/etc/htt
nobody   29463  0.5  1.1 2088 1304 ?        S 12:30:11  0:00 /usr/local/etc/htt
nobody   29431  0.5  1.0 2068 1284 ?        S 12:29:59  0:00 /usr/local/etc/htt
nobody   29466  0.5  1.0 2064 1280 ?        S 12:30:12  0:00 /usr/local/etc/htt
nobody   29452  0.5  1.1 2152 1368 ?        S 12:30:05  0:00 /usr/local/etc/htt
root         3  0.4  0.0    0    0 ?        S   Jan 22 35:25 fsflush
nobody   29447  0.4  1.0 2068 1284 ?        S 12:30:02  0:00 /usr/local/etc/htt
nobody   29461  0.4  1.0 2068 1264 ?        S 12:30:10  0:00 /usr/local/etc/htt
nobody   29390  0.4  1.1 2092 1308 ?        S 12:29:06  0:00 /usr/local/etc/htt
nobody   29467  0.4  1.0 2060 1280 ?        S 12:30:12  0:00 /usr/local/etc/htt
nobody   29474  0.4  1.0 2068 1284 ?        S 12:30:14  0:00 /usr/local/etc/htt
nobody   29449  0.4  1.1 2152 1368 ?        S 12:30:02  0:00 /usr/local/etc/htt
root     27423  0.4  1.1 1948 1392 ?        S 11:55:01  0:06 /usr/local/etc/htt
nobody   29415  0.4  1.0 2060 1276 ?        S 12:29:39  0:00 /usr/local/etc/htt
nobody   29464  0.4  1.0 2060 1272 ?        S 12:30:11  0:00 /usr/local/etc/htt
nobody   29444  0.4  1.0 2068 1284 ?        S 12:30:01  0:00 /usr/local/etc/htt
nobody   29468  0.4  1.1 2084 1300 ?        S 12:30:12  0:00 /usr/local/etc/htt
nobody   29404  0.4  1.0 2068 1284 ?        S 12:29:31  0:00 /usr/local/etc/htt
nobody   29413  0.4  1.1 2152 1340 ?        S 12:29:37  0:00 /usr/local/etc/htt
nobody   29477  0.4  1.0 2068 1260 ?        S 12:30:15  0:00 /usr/local/etc/htt
nobody   29405  0.4  1.0 2060 1276 ?        S 12:29:31  0:00 /usr/local/etc/htt
nobody   29476  0.4  1.0 2072 1264 ?        S 12:30:15  0:00 /usr/local/etc/htt
nobody   29407  0.4  1.1 2152 1368 ?        S 12:29:31  0:00 /usr/local/etc/htt
nobody   29479  0.4  1.0 2060 1276 ?        S 12:30:17  0:00 /usr/local/etc/htt
nobody   29430  0.3  1.0 2060 1276 ?        S 12:29:49  0:00 /usr/local/etc/htt
nobody   29410  0.3  1.1 2076 1292 ?        S 12:29:36  0:00 /usr/local/etc/htt
nobody   29496  0.3  1.0 2068 1284 ?        S 12:30:22  0:00 /usr/local/etc/htt
nobody   29402  0.3  1.0 2072 1288 ?        S 12:29:30  0:00 /usr/local/etc/htt
nobody   29422  0.3  1.0 2068 1284 ?        S 12:29:43  0:00 /usr/local/etc/htt
nobody   29478  0.3  1.0 2080 1272 ?        S 12:30:17  0:00 /usr/local/etc/htt
nobody   29491  0.3  1.0 2076 1288 ?        S 12:30:21  0:00 /usr/local/etc/htt
nobody   29499  0.3  1.1 2080 1292 ?        S 12:30:23  0:00 /usr/local/etc/htt
nobody   29493  0.3  1.0 2072 1280 ?        S 12:30:22  0:00 /usr/local/etc/htt
nobody   29469  0.3  1.0 2060 1276 ?        S 12:30:12  0:00 /usr/local/etc/htt
nobody   29391  0.3  1.0 2060 1276 ?        S 12:29:07  0:00 /usr/local/etc/htt
nobody   29481  0.3  1.0 2060 1272 ?        S 12:30:17  0:00 /usr/local/etc/htt
nobody   29408  0.2  1.1 2076 1292 ?        S 12:29:34  0:00 /usr/local/etc/htt
nobody   29446  0.2  1.0 2092 1280 ?        S 12:30:02  0:00 /usr/local/etc/htt
nobody   29424  0.2  1.0 2060 1276 ?        S 12:29:43  0:00 /usr/local/etc/htt
nobody   29401  0.2  1.4 3408 1756 ?        S 12:29:29  0:00 /usr/local/etc/htt
nobody   29400  0.1  1.1 2136 1352 ?        S 12:29:18  0:00 /usr/local/etc/htt
nobody   29500  0.1  0.7 1964  804 ?        S 12:30:23  0:00 /usr/local/etc/htt
root       178  0.1  0.7 1656  892 ?        S   Jan 22  0:02 /usr/sbin/nscd
nobody   29376  0.1  1.1 2080 1296 ?        S 12:28:47  0:00 /usr/local/etc/htt
nobody   29416  0.1  1.1 2080 1296 ?        S 12:29:40  0:00 /usr/local/etc/htt
nobody   29428  0.1  1.1 2164 1352 ?        S 12:29:43  0:00 /usr/local/etc/htt
nobody   29334  0.1  1.1 3408 1408 ?        S 12:27:57  0:00 /usr/local/etc/htt
nobody   29421  0.1  1.1 2128 1340 ?        S 12:29:42  0:00 /usr/local/etc/htt
root       172  0.1  0.6 1408  724 ?        S   Jan 22  0:16 /usr/sbin/cron
nobody   29423  0.1  1.1 2164 1368 ?        S 12:29:43  0:00 /usr/local/etc/htt
root     26827  0.1  0.6  924  752 pts/0    S 11:47:15  0:00 -csh
root         0  0.0  0.0    0    0 ?        T   Jan 22  0:00 sched
root         1  0.0  0.1  348  100 ?        S   Jan 22  0:08 /etc/init -
root         2  0.0  0.0    0    0 ?        S   Jan 22  0:00 pageout
root       113  0.0  0.4 1616  512 ?        S   Jan 22  0:00 /usr/sbin/rpcbind
root       115  0.0  0.0 1524    0 ?        S   Jan 22  0:00 /usr/sbin/keyserv
root       121  0.0  0.4 1680  448 ?        S   Jan 22  0:00 /usr/sbin/kerbd
root       130  0.0  0.4 1452  436 ?        S   Jan 22  0:00 /usr/sbin/in.named
root       134  0.0  0.7 1704  840 ?        S   Jan 22  0:03 /usr/sbin/inetd -s
root       137  0.0  0.4 1608  488 ?        S   Jan 22  0:00 /usr/lib/nfs/statd
root       139  0.0  0.4 1516  484 ?        S   Jan 22  0:00 /usr/lib/nfs/lockd
root       158  0.0  0.6 1784  736 ?        S   Jan 22  0:00 /usr/lib/autofs/au
root       162  0.0  0.6 1372  752 ?        S   Jan 22  0:00 /usr/sbin/syslogd
root       188  0.0  0.4 2472  444 ?        S   Jan 22  0:00 /usr/lib/lpsched
root       198  0.0  0.5 1344  552 ?        S   Jan 22  0:00 lpNet
root       200  0.0  0.3  748  396 ?        S   Jan 22  0:00 /usr/lib/utmpd
root       202  0.0  0.7 1952  852 ?        S   Jan 22  0:00 /usr/sbin/vold
root       240  0.0  0.5 1516  608 ?        S   Jan 22  0:00 /usr/lib/nfs/nfsd 
root       242  0.0  0.5 1664  588 ?        S   Jan 22  0:00 /usr/lib/nfs/mount
root       251  0.0  0.5 1296  652 ?        S   Jan 22  0:00 /usr/lib/saf/sac -
root       252  0.0  0.5 1352  604 console  S   Jan 22  0:00 /usr/lib/saf/ttymo
root       254  0.0  0.6 1368  668 ?        S   Jan 22  0:00 /usr/lib/saf/ttymo
root     26825  0.0  0.8 1352  936 ?        S 11:47:14  0:00 in.telnetd
nobody   29247  0.0  1.0 2068 1284 ?        S 12:26:02  0:00 /usr/local/etc/htt
nobody   29337  0.0  1.0 2068 1284 ?        S 12:28:04  0:00 /usr/local/etc/htt
nobody   29351  0.0  1.1 2160 1376 ?        S 12:28:16  0:00 /usr/local/etc/htt


VMSTAT
 procs     memory            page            disk          faults      cpu
 r b w   swap  free  re  mf pi po fr de sr f0 s0 s1 --   in   sy   cs us sy id
 0 0 0  43536 15488   0 118  2  0  3  0  0  0  0  3  0  408  649  525 10 26 63
 0 0 0 177920  6748   0 426  0  0  0  0  0  0  0  0  0  499  945  767  2 47 51
 3 0 0 175208  5232   0 1086 0  0  0  0  0  0  0  0  0  504 1431  901 14 72 14
 1 0 0 175120  5084   0 214  0  0  0  0  0  0  0  0  0  526  926  744 12 34 54
 4 0 0 168056  2752   0 1206 0  0  0  0  0  0  0  1  0  561 1489  999  5 83 12 

VMSTAT after the webserver all processes are killed.
 procs     memory            page            disk          faults      cpu
 r b w   swap  free  re  mf pi po fr de sr f0 s0 s1 --   in   sy   cs us sy id
 0 0 0  43576 15484   0 118  2  0  3  0  0  0  0  3  0  408  649  525 10 26 63
 0 0 0 208088 23828   0   7  0  0  0  0  0  0  0  1  0  275   13  339  0  4 96
 0 0 0 208088 23828   0   0  0  0  0  0  0  0  0  0  0  244   10  345  0  6 94
 0 0 0 208088 23828   0   0  0  0  0  0  0  0  0  0  0  245   10  322  0  6 94
 0 0 0 208088 23828   0   0  0  0  0  0  0  0  0  0  0  225   10  284  0  8 92
 0 0 0 208088 23828   0   0  0  0  0  0  0  0  0  0  0  243   10  293  0  6 94 

             after a reboot (and webserver is running)

VMSTAT
 procs     memory            page            disk          faults      cpu

 r b w   swap  free  re  mf pi po fr de sr f0 s0 s1 --   in   sy   cs us sy id

 0 0 0 183196 72892   0  28 64  0  0  0  0  0  0 11  0  573  930  727 10 37 53

 0 0 0 183192 72808   0  16 72  0  0  0  0  0  1  8  0  536  814  647  2 35 63

 4 0 0 183728 73076   0  25 32  0  0  0  0  0  0  5  0  475 1047  697  8 39 53

 0 0 0 183724 73012   0  13 72  0  0  0  0  0  0  8  0  464  606  495  6 22 72

 0 0 0 184316 73316   0  24  0  0  0  0  0  0  0  1  0  384  714  574  6 22 72


PS -AUX

USER       PID %CPU %MEM   SZ  RSS TT       S    START  TIME COMMAND

root       336  2.5  0.9 1288 1104 pts/1    O 12:36:45  0:00 /usr/ucb/ps -aux

nobody     299  0.6  1.0 2080 1284 ?        S 12:35:47  0:00 /usr/local/etc/htt

nobody     274  0.5  1.0 2072 1276 ?        S 12:35:43  0:00 /usr/local/etc/htt

nobody     305  0.5  1.1 2088 1292 ?        S 12:35:48  0:00 /usr/local/etc/htt

nobody     283  0.5  1.1 2100 1304 ?        S 12:35:45  0:00 /usr/local/etc/htt

nobody     270  0.5  1.0 2076 1284 ?        S 12:35:42  0:00 /usr/local/etc/htt

nobody     281  0.5  1.0 2068 1272 ?        S 12:35:45  0:00 /usr/local/etc/htt

nobody     296  0.5  1.1 2092 1296 ?        S 12:35:46  0:00 /usr/local/etc/htt

nobody     280  0.5  1.0 2068 1272 ?        S 12:35:44  0:00 /usr/local/etc/htt

nobody     298  0.5  1.1 2104 1308 ?        S 12:35:47  0:00 /usr/local/etc/htt

nobody     282  0.5  1.0 2060 1268 ?        S 12:35:45  0:00 /usr/local/etc/htt

nobody     301  0.5  1.0 2084 1288 ?        S 12:35:47  0:00 /usr/local/etc/htt

nobody     229  0.5  1.1 2092 1300 ?        S 12:35:37  0:00 /usr/local/etc/htt

nobody     302  0.5  1.0 2076 1280 ?        S 12:35:47  0:00 /usr/local/etc/htt

nobody     300  0.5  1.1 2092 1296 ?        S 12:35:47  0:00 /usr/local/etc/htt

nobody     279  0.4  1.0 2064 1268 ?        S 12:35:44  0:00 /usr/local/etc/htt

nobody     284  0.4  1.0 2084 1288 ?        S 12:35:45  0:00 /usr/local/etc/htt

nobody     304  0.4  1.1 2092 1296 ?        S 12:35:48  0:00 /usr/local/etc/htt

nobody     292  0.4  1.0 2076 1280 ?        S 12:35:45  0:00 /usr/local/etc/htt

nobody     337  0.4  1.0 2072 1244 ?        S 12:36:45  0:00 /usr/local/etc/htt

nobody     289  0.3  1.0 2068 1272 ?        S 12:35:45  0:00 /usr/local/etc/htt

nobody     287  0.3  1.1 2152 1356 ?        S 12:35:45  0:00 /usr/local/etc/htt

nobody     335  0.3  1.0 2068 1268 ?        S 12:36:39  0:00 /usr/local/etc/htt

nobody     272  0.3  1.1 2132 1336 ?        S 12:35:42  0:00 /usr/local/etc/htt

nobody     275  0.3  1.0 2080 1284 ?        S 12:35:43  0:00 /usr/local/etc/htt

nobody     223  0.3  1.0 2072 1272 ?        S 12:35:37  0:00 /usr/local/etc/htt

nobody     306  0.3  1.0 2068 1272 ?        S 12:35:48  0:00 /usr/local/etc/htt

nobody     291  0.3  1.0 2068 1272 ?        S 12:35:45  0:00 /usr/local/etc/htt

nobody     294  0.3  1.0 2060 1264 ?        S 12:35:46  0:00 /usr/local/etc/htt

nobody     231  0.3  1.0 2060 1268 ?        S 12:35:38  0:00 /usr/local/etc/htt

root         3  0.2  0.0    0    0 ?        S 12:34:45  0:00 fsflush

nobody     285  0.2  1.1 2136 1336 ?        S 12:35:45  0:00 /usr/local/etc/htt

nobody     276  0.2  1.0 2068 1272 ?        S 12:35:43  0:00 /usr/local/etc/htt

nobody     268  0.2  1.1 2160 1364 ?        S 12:35:41  0:00 /usr/local/etc/htt

root       329  0.2  0.4  708  528 pts/0    S 12:35:58  0:00 script -a /out

nobody     269  0.2  1.0 2072 1276 ?        S 12:35:42  0:00 /usr/local/etc/htt

nobody     290  0.2  1.1 2192 1396 ?        S 12:35:45  0:00 /usr/local/etc/htt

root       218  0.1  1.1 1948 1344 ?        S 12:35:37  0:00 /usr/local/etc/htt

root       330  0.1  0.4  732  456 pts/0    S 12:35:58  0:00 script -a /out

root       331  0.1  0.6  924  708 pts/1    S 12:35:58  0:00 sh -i

root       172  0.1  0.6  924  720 pts/0    S 12:35:28  0:00 -csh

root       205  0.1  1.1 1952 1380 ?        S 12:35:32  0:00 /usr/sbin/vold

root       252  0.1  0.8 1664  980 ?        S 12:35:39  0:00 /usr/lib/nfs/mount

root       134  0.0  0.9 1704 1104 ?        S 12:35:16  0:00 /usr/sbin/inetd -s

root       250  0.0  0.7 1516  860 ?        S 12:35:38  0:00 /usr/lib/nfs/nfsd 

root       264  0.0  0.7 1368  904 ?        S 12:35:40  0:00 /usr/lib/saf/ttymo

root       139  0.0  0.6 1516  740 ?        S 12:35:17  0:00 /usr/lib/nfs/lockd

root       162  0.0  0.7 1372  836 ?        S 12:35:18  0:00 /usr/sbin/syslogd

nobody     230  0.0  1.1 2124 1324 ?        S 12:35:38  0:00 /usr/local/etc/htt

root       261  0.0  0.7 1296  832 ?        S 12:35:39  0:00 /usr/lib/saf/sac -

root       262  0.0  0.7 1352  864 console  S 12:35:39  0:00 /usr/lib/saf/ttymo

root       170  0.0  0.8 1388  976 ?        S 12:35:26  0:00 in.telnetd

root       202  0.0  0.7 1344  872 ?        S 12:35:32  0:00 lpNet

root         1  0.0  0.2  348  180 ?        S 12:34:45  0:00 /etc/init -

root       113  0.0  0.5 1616  608 ?        S 12:35:09  0:00 /usr/sbin/rpcbind

root       137  0.0  0.7 1608  848 ?        S 12:35:16  0:00 /usr/lib/nfs/statd

root       158  0.0  0.9 1784 1140 ?        S 12:35:18  0:00 /usr/lib/autofs/au

root       191  0.0  0.6 2472  780 ?        S 12:35:31  0:00 /usr/lib/lpsched

root         0  0.0  0.0    0    0 ?        T 12:34:45  0:00 sched

root         2  0.0  0.0    0    0 ?        S 12:34:45  0:00 pageout

root       115  0.0  0.1 1524   24 ?        S 12:35:09  0:00 /usr/sbin/keyserv

root       121  0.0  0.7 1680  864 ?        S 12:35:10  0:00 /usr/sbin/kerbd

root       130  0.0  0.6 1420  656 ?        S 12:35:16  0:00 /usr/sbin/in.named

root       175  0.0  0.6 1320  724 ?        S 12:35:29  0:00 /usr/sbin/cron

root       181  0.0  0.8 1608  972 ?        S 12:35:30  0:00 /usr/sbin/nscd

root       203  0.0  0.4  748  420 ?        S 12:35:32  0:00 /usr/lib/utmpd

nobody     273  0.0  0.0    0    0          Z           0:00  <defunct>

nobody     277  0.0  0.0    0    0          Z           0:00  <defunct>

nobody     303  0.0  0.0    0    0          Z           0:00  <defunct>
>How-To-Repeat:

>Fix:

>Audit-Trail:
>Unformatted:
[In order for any reply to be added to the PR database, ]
[you need to include <apbugs@Apache.Org> in the Cc line ]
[and leave the subject line UNCHANGED.  This is not done]
[automatically because of the potential for mail loops. ]




Mime
View raw message