Mailing-List: contact users-help@httpd.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@httpd.apache.org
Received-SPF: pass (asf.osuosl.org: local policy)
Message-ID: <63625.24.90.33.115.1119234760.squirrel@24.90.33.115>
Date: Sun, 19 Jun 2005 22:32:40 -0400 (EDT)
From: "kalin mintchev" <kalin@el.net>
To: users@httpd.apache.org
User-Agent: SquirrelMail/1.4.3a
MIME-Version: 1.0
Content-Type: text/plain;charset=iso-8859-1
Content-Transfer-Encoding: 8bit
Importance: Normal
References: <62198.24.90.33.115.1118163726.squirrel@24.90.33.115>   
    <200506072020.40962.mymaillists@gmx.at>
In-Reply-To: <200506072020.40962.mymaillists@gmx.at>
Subject: Re: [users@httpd] weird httpd processes

hi all...

i started this a few weeks ago.. the situation hasn't been improving...
i'm getting really frustrated. i've looked for scripts but i can not find
anything wrong. besides there are about 150 domains on that machine and i
can't go through each php or pearl file....

i woke up today to about 20 or so stalled processes - the most ever. i do
have server-status on but that just points to the machine name...  what to
do? i can't shut down domains until i figured where the hell is the
offending script....

right now i have one process running wild but top and server-status report
differnte cpu usage for those processes. here:

from top:

55 processes:  2 running, 53 sleeping
CPU states: 16.0% user,  0.0% nice, 83.7% system,  0.4% interrupt,  0.0% idle
Mem: 349M Active, 428M Inact, 177M Wired, 38M Cache, 112M Buf, 9944K Free
Swap: 2032M Total, 100K Used, 2032M Free

  PID USERNAME      PRI NICE  SIZE    RES STATE    TIME   WCPU    CPU COMMAND
66696 nobody         59   0 16168K 10580K RUN    153:54 98.24% 98.24% httpd
36782 qtss            2   0  6324K  4980K poll    59:51  0.00%  0.00%
DarwinStr

from server-status:
Srv	PID	Acc	M	CPU	SS	Req	Conn	Child	Slot	Host	VHost	Request
0-0	66687	0/4206/4206	_ 	7.34	3	22	0.0	114.66	114.66
	65.214.44.36	richardedwardsnyc.com	GET /s_s_singles.w.02/09.html HTTP/1.0
1-0	66688	0/4341/4341	_ 	9.20	0	0	0.0	111.73	111.73
	24.149.136.172	fotofolio.com	GET /images/blbackbttn.gif HTTP/1.1
2-0	66689	11/4136/4136	K 	7.63	0	0	0.0	85.68	85.68
	68.111.73.32	TimesSquare.com	GET
/cut03/images/homecut/solo/frame_r4_c6.gif HTTP/1.1
3-0	66690	1/4426/4426	K 	9.09	10	0	0.0	108.92	108.92
	24.185.235.98	oldcove.org	GET /images/juniorsailing.gif HTTP/1.1
4-0	66691	1/4416/4416	K 	9.58	10	0	0.0	108.34	108.34
	24.185.235.98	oldcove.org	GET /images/T-juniorsailing.gif HTTP/1.1
5-0	66692	1/4552/4552	K 	7.52	10	0	0.0	83.19	83.19
	24.185.235.98	oldcove.org	GET /images/spacer.gif HTTP/1.1
6-0	66693	0/4286/4286	W 	8.07	111	0	0.0	122.70	122.70
	86.127.1.145	giganticmusic.com	GET /home.htm HTTP/1.1
7-0	66694	1/4737/4737	K 	9.10	25	7817	99.8	77.52	77.52
	68.111.73.32	TimesSquare.com	POST /cut03/allResultsT.php HTTP/1.1
8-0	66695	0/4251/4251	_ 	9.89	11	303	0.0	143.56	143.56
	209.212.80.98	giganticmusic.com	GET /smoker.swf HTTP/1.1
9-0	66696	0/3151/3151	W 	6.50	9007	0	0.0	62.23	62.23
	207.214.244.4	guey.el.net	GET / HTTP/1.0
10-0	70405	0/1286/3848	_ 	2.93	12	0	0.0	26.16	102.67
	209.212.80.98	giganticmusic.com	GET /favicon.ico HTTP/1.1
11-0	71906	11/22/3243	K 	0.00	0	0	0.0	0.08	87.98
	68.111.73.32	TimesSquare.com	GET /cut03/images/rank2.0.gif HTTP/1.1
12-0	71921	12/14/2890	K 	0.00	0	0	0.0	0.02	42.80
	68.111.73.32	TimesSquare.com	GET /cut03/images/rank2.5.gif HTTP/1.1
13-0	71923	14/14/2392	W 	0.00	0	0	257.7	0.25	67.16
	209.212.80.98	giganticmusic.com	GET /images/shows/kjhlkjh.gif HTTP/1.1
14-0	71924	5/5/2025	K 	0.00	0	0	15.7	0.02	22.99
	209.212.80.98	giganticmusic.com	GET
/images/artists/someaction/sacover.jpg HTTP/1.1
15-0	71925	2/2/1634	W 	0.00	4	0	20.4	0.02	37.66
	24.90.33.115	guey.el.net	GET /server-status HTTP/1.1

etc..

i'd really appreciate some help...   thanks....


> Hi,
>
> This looks like a script that is running out of control.  We've had similar
> things here, except that our problem was the script writer failed to
close
> the mysql connections.  The user was waiting for data which never came, hit
> cancel, then tried again.  The scripts used pconnect calls.  This
brought
> a
> 30,000 Euro server to its knees several times by eating all the
available
> sockets, at least until we found out what the problem was.
>
> In your case, set the max execution time for scripts to something short
like
> 10 seconds.  Consider also disabling ini_set because we also had
problems
> with users playing with our settings and stressing our server.  Be sure
that
> all uri includes are disabled too (XSS attack problems).
>
> To find the exact script is difficult.  Something to note is your CPU usage
> has 65% system.  Look for scripts that were uploaded around the times
your
> server started having problems, then try to run them is one possibility
to
> find out what's happening.  Look in the databases for connections that
are
> open and idle.  Also, if your server comes down off its high load, look
in
> the apache log files to see which scripts just completed.  Unfortunately
apache writes to the logs only after the script or data has been
(un)successfully sent.
>
> Markus
>
>
>
> On Tuesday 07 June 2005 19:02, kalin mintchev wrote:
>> hi all...
>> for the last 4 - 5 days i noticed something weird on one of the freebsd
boxes here. one httpd process -or sometimes 2 or 3 - take up all the
cpu.
>> and it looks like this:
> 8<---
>
> ---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server
Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
>    "   from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>


-- 
--
"In a world without walls or fences, who needs Windows and Gates?"


---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org