Return-Path: Delivered-To: apmail-httpd-users-archive@www.apache.org Received: (qmail 18983 invoked from network); 17 Dec 2009 02:18:12 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 17 Dec 2009 02:18:12 -0000 Received: (qmail 48763 invoked by uid 500); 17 Dec 2009 02:18:09 -0000 Delivered-To: apmail-httpd-users-archive@httpd.apache.org Received: (qmail 48705 invoked by uid 500); 17 Dec 2009 02:18:09 -0000 Mailing-List: contact users-help@httpd.apache.org; run by ezmlm Precedence: bulk Reply-To: users@httpd.apache.org list-help: list-unsubscribe: List-Post: List-Id: Delivered-To: mailing list users@httpd.apache.org Received: (qmail 48695 invoked by uid 99); 17 Dec 2009 02:18:09 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 17 Dec 2009 02:18:09 +0000 X-ASF-Spam-Status: No, hits=-2.6 required=5.0 tests=BAYES_00,HTML_MESSAGE X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of raliste@gmail.com designates 209.85.210.189 as permitted sender) Received: from [209.85.210.189] (HELO mail-yx0-f189.google.com) (209.85.210.189) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 17 Dec 2009 02:18:06 +0000 Received: by yxe27 with SMTP id 27so1588739yxe.10 for ; Wed, 16 Dec 2009 18:17:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :from:date:message-id:subject:to:content-type; bh=9LTFJtEuo/kKBiv+lFqH5OlV9OKemTVItpRyQwSdDEM=; b=luLQgv08Re0ObSmWGASs4zxoMaMOSylubSh45VnPF6ocm1lMrKHH56mr9wQ4DkSclV 6tOOqG3sD5Fpx9nxZbDFuaSV/asUXbQNQBLgOkfFoHlKO2/xGT4+7h0n6Gp/zUuc06L7 ptmNgKDYGkbYXm09OIkJSQQoi1GMepA9s9ags= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=gQ9U205XSIv46Ptcic5QanMm3NXZ50hhop0CtQql/KfTGMSxFg75rPQbbmzko22NMU QjP4BplIILP214IY31pLp+XPjbXoUxy5JH/a9J6XKgsoxKNVnPLa+++7eB/PJNCGrDlt ba6f0juu7xctdWtQu46lY3jc/XdffoeY+Fl28= MIME-Version: 1.0 Received: by 10.150.9.17 with SMTP id 17mr2944357ybi.252.1261016265062; Wed, 16 Dec 2009 18:17:45 -0800 (PST) In-Reply-To: <93e82a90912161813q5dfd1d5fjb7c0b55a5093ef9d@mail.gmail.com> References: <2e027be00912160828y1f265f5eoc9f5710f2a2e1b5f@mail.gmail.com> <93e82a90912161813q5dfd1d5fjb7c0b55a5093ef9d@mail.gmail.com> From: "Rodrigo Aliste P." Date: Wed, 16 Dec 2009 23:17:25 -0300 Message-ID: <93e82a90912161817u28537e87m1037ba5c408abd10@mail.gmail.com> To: users@httpd.apache.org Content-Type: multipart/alternative; boundary=000e0cd7618088ff0a047ae33ade Subject: Re: [users@httpd] Apache intermittent crash - HELP!!! --000e0cd7618088ff0a047ae33ade Content-Type: text/plain; charset=ISO-8859-1 I just noticed that your error log shows: [Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting, consider raising the MaxClients setting so try adjusting your MaxClient setting and your prefork settings. ( http://httpd.apache.org/docs/2.0/mod/prefork.html) I'm not an expert on tuning those settings, but I'm sure someone else here could help you. 2009/12/16 Rodrigo Aliste P. > Gary: > > The same thing happened to one of my servers. After long research I got to > the conclusion that the problem was caused by the XCache module. If you have > it, try to disable it. > > > Rodrigo > > 2009/12/16 gary lim > > Hi, >> >> 1. Unfortunately, our website depend heavily on PHP to operate, so it not >> possible for us to disable it. We also highly suspect that the crash is >> related to one of our php module because we are running a lot of Joomla >> component. >> >> 2. unfortunately, we are unable to track which module is causing this >> issue. Any idea how we can check which script is causing the problem because >> this has been going on for weeks and we are still unable to determine the >> source. >> >> 3. Kindly advise how we can perform the migration of our existing codebase >> to FastCGI/SCGI without breaking the current website? >> >> >> rgds >> >> >> On Thu, Dec 17, 2009 at 12:28 AM, Tom Evans wrote: >> >>> On Wed, Dec 16, 2009 at 4:20 PM, gary lim wrote: >>> > >>> > Dear all, >>> > >>> > 1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium(R) 4 >>> CPU 3.00GHz GNU/Linux >>> > 2. The below info are my machine config and log output >>> > 3. Kindly advise how to resolve intermittent apache crash even after >>> performance tuning >>> > >>> > httpd.conf output >>> > # prefork MPM [DEFAULT IF USE=-threads] >>> > # StartServers: number of server processes to start >>> > # MinSpareServers: minimum number of server processes which are kept >>> spare >>> > # MaxSpareServers: maximum number of server processes which are kept >>> spare >>> > # MaxClients: maximum number of server processes allowed to start >>> > # MaxRequestsPerChild: maximum number of requests a server process >>> serves >>> > >>> > StartServers 2 >>> > MinSpareServers 4 >>> > MaxSpareServers 6 >>> > MaxClients 100 >>> > MaxRequestsPerChild 500 >>> > >>> > >>> > # worker MPM [DEFAULT IF USE=threads] >>> > # StartServers: initial number of server processes to start >>> > # MaxClients: maximum number of simultaneous client connections >>> > # MinSpareThreads: minimum number of worker threads which are kept >>> spare >>> > # MaxSpareThreads: maximum number of worker threads which are kept >>> spare >>> > # ThreadsPerChild: constant number of worker threads in each server >>> process >>> > # MaxRequestsPerChild: maximum number of requests a server process >>> serves >>> > >>> > StartServers 2 >>> > MaxClients 100 >>> > MinSpareThreads 25 >>> > MaxSpareThreads 75 >>> > ThreadsPerChild 25 >>> > MaxRequestsPerChild 500 >>> > >>> > >>> > >>> > >>> > top output >>> > top - 00:09:00 up 157 days, 23:12, 2 users, load average: 0.67, 0.85, >>> 0.80 >>> > Tasks: 47 total, 1 running, 46 sleeping, 0 stopped, 0 zombie >>> > Cpu(s): 8.3% us, 1.4% sy, 0.0% ni, 88.0% id, 2.2% wa, 0.0% hi, >>> 0.0% si >>> > Mem: 917652k total, 858880k used, 58772k free, 142820k >>> buffers >>> > Swap: 1048568k total, 37844k used, 1010724k free, 434476k cached >>> > >>> > apache2 error log >>> > *** glibc detected *** /usr/sbin/apache2: double free or corruption >>> (fasttop): 0x08b4faf0 *** >>> > [Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting, >>> consider raising the MaxClients setting >>> > *** glibc detected *** /usr/sbin/apache2: double free or corruption >>> (fasttop): 0x086a1170 *** >>> > *** glibc detected *** /usr/sbin/apache2: double free or corruption >>> (fasttop): 0x08b48c08 *** >>> > [Wed Dec 16 23:53:07 2009] [warn] child process 1325 still did not >>> exit, sending a SIGTERM >>> > [Wed Dec 16 23:53:07 2009] [warn] child process 19910 still did not >>> exit, sending a SIGTERM >>> > [Wed Dec 16 23:53:07 2009] [warn] child process 10569 still did not >>> exit, sending a SIGTERM >>> > [Wed Dec 16 23:53:07 2009] [warn] child process 16964 still did not >>> exit, sending a SIGTERM >>> > [Wed Dec 16 23:53:07 2009] [warn] child process 19718 still did not >>> exit, sending a SIGTERM >>> > [Wed Dec 16 23:53:09 2009] [warn] child process 1325 still did not >>> exit, sending a SIGTERM >>> > [Wed Dec 16 23:53:09 2009] [warn] child process 19910 still did not >>> exit, sending a SIGTERM >>> > [Wed Dec 16 23:53:09 2009] [warn] child process 10569 still did not >>> exit, sending a SIGTERM >>> > [Wed Dec 16 23:53:09 2009] [warn] child process 16964 still did not >>> exit, sending a SIGTERM >>> > [Wed Dec 16 23:53:09 2009] [warn] child process 19718 still did not >>> exit, sending a SIGTERM >>> > [Wed Dec 16 23:53:11 2009] [warn] child process 1325 still did not >>> exit, sending a SIGTERM >>> > [Wed Dec 16 23:53:11 2009] [warn] child process 19910 still did not >>> exit, sending a SIGTERM >>> > [Wed Dec 16 23:53:11 2009] [warn] child process 10569 still did not >>> exit, sending a SIGTERM >>> > [Wed Dec 16 23:53:11 2009] [warn] child process 16964 still did not >>> exit, sending a SIGTERM >>> > [Wed Dec 16 23:53:11 2009] [warn] child process 19718 still did not >>> exit, sending a SIGTERM >>> > [Wed Dec 16 23:53:13 2009] [error] child process 1325 still did not >>> exit, sending a SIGKILL >>> > [Wed Dec 16 23:53:13 2009] [error] child process 19910 still did not >>> exit, sending a SIGKILL >>> > [Wed Dec 16 23:53:13 2009] [error] child process 10569 still did not >>> exit, sending a SIGKILL >>> > [Wed Dec 16 23:53:13 2009] [error] child process 16964 still did not >>> exit, sending a SIGKILL >>> > >>> > >>> > Joomla / PHP / Apache >>> > System Information >>> > PHP built On: Linux trillian 2.6.16.29-ab1-xenU #1 SMP Sat Sep 23 >>> 19:20:15 SGT 2006 i686 >>> > Database Version: 5.0.24a-log >>> > PHP Version: 5.1.6-pl4-gentoo >>> > Web Server: Apache >>> > WebServer to PHP interface: apache2handler >>> > Joomla! Version: Joomla! 1.0.15 Stable [ Daytime ] 22 February 2008 >>> 23:00 UTC >>> > User Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; >>> rv:1.9.0.15) Gecko/2009101601 Firefox/3.0.15 (.NET CLR 3.5.30729) >>> > >>> > Mysql >>> > mysql Ver 14.12 Distrib 5.0.24a, for pc-linux-gnu (i686) using >>> readline 5.1 >>> > >>> >>> Does it work if you disable PHP? Does it still exhibit problems if you >>> run PHP out of process, eg using fastcgi or scgi (can PHP talk SCGI?). >>> For my money, the double free()s are coming from PHP or a PHP module. >>> >>> Cheers >>> >>> Tom >>> >>> --------------------------------------------------------------------- >>> The official User-To-User support forum of the Apache HTTP Server >>> Project. >>> See for more info. >>> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org >>> " from the digest: users-digest-unsubscribe@httpd.apache.org >>> For additional commands, e-mail: users-help@httpd.apache.org >>> >>> >> > > > -- > Rodrigo Aliste P. > > -- Rodrigo Aliste P. --000e0cd7618088ff0a047ae33ade Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable I just noticed that your error log shows:

[Tue Dec 15 13:04:22 2009= ] [error] server reached MaxClients setting, consider raising the MaxClient= s setting

so try adjusting your MaxClient setting and your prefork s= ettings. (htt= p://httpd.apache.org/docs/2.0/mod/prefork.html)

I'm not an expert on tuning those settings, but I'm sure someon= e else here could help you.


2009/12/1= 6 Rodrigo Aliste P. <raliste@gmail.com>
Gary:

The = same thing happened to one of my servers. After long research I got to the = conclusion that the problem was caused by the XCache module. If you have it= , try to disable it.


Rodrigo

2009/12/16 gary lim <garylim.in@gmail.com>

Hi,

1. Unfortunately, our website depend heavily on PHP to operate, = so it not possible for us to disable it. We also highly suspect that the cr= ash is related to one of our php module because we are running a lot of Joo= mla component.

2. unfortunately, we are unable to track which module is causing this i= ssue. Any idea how we can check which script is causing the problem because= this has been going on for weeks and we are still unable to determine the = source.

3. Kindly advise how we can perform the migration of our existing codeb= ase to FastCGI/SCGI without breaking the current website?


rgds

On Thu, Dec 17, 2009 at 12:28 AM, Tom = Evans <= tevans.uk@googlemai= l.com> wrote:
<= div>On Wed, Dec 16, 2009 at 4:20 PM, gary lim <garylim.in@gmail.com> wrote:
>
> Dear all,
>
> 1. I'm using Linux 2.6.16.29 and running on i686 Intel(R) Pentium(= R) 4 CPU 3.00GHz GNU/Linux
> 2. The below info are my machine config and log output
> 3. Kindly advise how to resolve intermittent apache crash even after p= erformance tuning
>
> httpd.conf output
> # prefork MPM [DEFAULT IF USE=3D-threads]
> # StartServers: number of server processes to start
> # MinSpareServers: minimum number of server processes which are kept s= pare
> # MaxSpareServers: maximum number of server processes which are kept s= pare
> # MaxClients: maximum number of server processes allowed to start
> # MaxRequestsPerChild: maximum number of requests a server process ser= ves
> <IfModule prefork.c>
> =A0=A0=A0 StartServers=A0=A0=A0=A0=A0=A0=A0=A0 2
> =A0=A0=A0 MinSpareServers=A0=A0=A0=A0=A0 4
> =A0=A0=A0 MaxSpareServers=A0=A0=A0=A0 6
> =A0=A0=A0 MaxClients=A0=A0=A0=A0=A0=A0=A0=A0 100
> =A0=A0=A0 MaxRequestsPerChild=A0 500
> </IfModule>
>
> # worker MPM [DEFAULT IF USE=3Dthreads]
> # StartServers: initial number of server processes to start
> # MaxClients: maximum number of simultaneous client connections
> # MinSpareThreads: minimum number of worker threads which are kept spa= re
> # MaxSpareThreads: maximum number of worker threads which are kept spa= re
> # ThreadsPerChild: constant number of worker threads in each server pr= ocess
> # MaxRequestsPerChild: maximum number of requests a server process ser= ves
> <IfModule worker.c>
> =A0=A0=A0 StartServers=A0=A0=A0=A0=A0=A0=A0=A0 2
> =A0=A0=A0 MaxClients=A0=A0=A0=A0=A0=A0=A0=A0 100
> =A0=A0=A0 MinSpareThreads=A0=A0=A0=A0 25
> =A0=A0=A0 MaxSpareThreads=A0=A0=A0=A0 75
> =A0=A0=A0 ThreadsPerChild=A0=A0=A0=A0 25
> =A0=A0=A0 MaxRequestsPerChild=A0 500
> </IfModule>
>
>
>
> top output
> top - 00:09:00 up 157 days, 23:12,=A0 2 users,=A0 load average: 0.67, = 0.85, 0.80
> Tasks:=A0 47 total,=A0=A0 1 running,=A0 46 sleeping,=A0=A0 0 stopped,= =A0=A0 0 zombie
> Cpu(s):=A0 8.3% us,=A0 1.4% sy,=A0 0.0% ni, 88.0% id,=A0 2.2% wa,=A0 0= .0% hi,=A0 0.0% si
> Mem:=A0=A0=A0 917652k total,=A0=A0 858880k used,=A0=A0=A0 58772k free,= =A0=A0 142820k buffers
> Swap:=A0 1048568k total,=A0=A0=A0 37844k used,=A0 1010724k free,=A0=A0= 434476k cached
>
> apache2 error log
> *** glibc detected *** /usr/sbin/apache2: double free or corruption (f= asttop): 0x08b4faf0 ***
> [Tue Dec 15 13:04:22 2009] [error] server reached MaxClients setting, = consider raising the MaxClients setting
> *** glibc detected *** /usr/sbin/apache2: double free or corruption (f= asttop): 0x086a1170 ***
> *** glibc detected *** /usr/sbin/apache2: double free or corruption (f= asttop): 0x08b48c08 ***
> [Wed Dec 16 23:53:07 2009] [warn] child process 1325 still did not exi= t, sending a SIGTERM
> [Wed Dec 16 23:53:07 2009] [warn] child process 19910 still did not ex= it, sending a SIGTERM
> [Wed Dec 16 23:53:07 2009] [warn] child process 10569 still did not ex= it, sending a SIGTERM
> [Wed Dec 16 23:53:07 2009] [warn] child process 16964 still did not ex= it, sending a SIGTERM
> [Wed Dec 16 23:53:07 2009] [warn] child process 19718 still did not ex= it, sending a SIGTERM
> [Wed Dec 16 23:53:09 2009] [warn] child process 1325 still did not exi= t, sending a SIGTERM
> [Wed Dec 16 23:53:09 2009] [warn] child process 19910 still did not ex= it, sending a SIGTERM
> [Wed Dec 16 23:53:09 2009] [warn] child process 10569 still did not ex= it, sending a SIGTERM
> [Wed Dec 16 23:53:09 2009] [warn] child process 16964 still did not ex= it, sending a SIGTERM
> [Wed Dec 16 23:53:09 2009] [warn] child process 19718 still did not ex= it, sending a SIGTERM
> [Wed Dec 16 23:53:11 2009] [warn] child process 1325 still did not exi= t, sending a SIGTERM
> [Wed Dec 16 23:53:11 2009] [warn] child process 19910 still did not ex= it, sending a SIGTERM
> [Wed Dec 16 23:53:11 2009] [warn] child process 10569 still did not ex= it, sending a SIGTERM
> [Wed Dec 16 23:53:11 2009] [warn] child process 16964 still did not ex= it, sending a SIGTERM
> [Wed Dec 16 23:53:11 2009] [warn] child process 19718 still did not ex= it, sending a SIGTERM
> [Wed Dec 16 23:53:13 2009] [error] child process 1325 still did not ex= it, sending a SIGKILL
> [Wed Dec 16 23:53:13 2009] [error] child process 19910 still did not e= xit, sending a SIGKILL
> [Wed Dec 16 23:53:13 2009] [error] child process 10569 still did not e= xit, sending a SIGKILL
> [Wed Dec 16 23:53:13 2009] [error] child process 16964 still did not e= xit, sending a SIGKILL
>
>
> Joomla / PHP / Apache
> System Information
> PHP built On: Linux trillian 2.6.16.29-ab1-xenU #1 SMP Sat Sep 23 19:2= 0:15 SGT 2006 i686
> Database Version: 5.0.24a-log
> PHP Version: 5.1.6-pl4-gentoo
> Web Server: Apache
> WebServer to PHP interface: apache2handler
> Joomla! Version: Joomla! 1.0.15 Stable [ Daytime ] 22 February 2008 23= :00 UTC
> User Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1= 5) Gecko/2009101601 Firefox/3.0.15 (.NET CLR 3.5.30729)
>
> Mysql
> mysql=A0 Ver 14.12 Distrib 5.0.24a, for pc-linux-gnu (i686) using read= line 5.1
>

Does it work if you disable PHP? Does it still exhibit problems= if you
run PHP out of process, eg using fastcgi or scgi (can PHP talk SCGI?).
For my money, the double free()s are coming from PHP or a PHP module.

Cheers

Tom

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.<= br> See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
=A0 " =A0 from the digest: users-digest-unsubscribe@httpd.apache.= org
For additional commands, e-mail: users-help@httpd.apache.org





--
Rodrigo Aliste P.




--
Rodrigo Aliste P= .

--000e0cd7618088ff0a047ae33ade--