Return-Path: X-Original-To: apmail-perl-modperl-archive@www.apache.org Delivered-To: apmail-perl-modperl-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C8C7DEC20 for ; Tue, 8 Jan 2013 14:05:41 +0000 (UTC) Received: (qmail 92014 invoked by uid 500); 8 Jan 2013 14:05:40 -0000 Delivered-To: apmail-perl-modperl-archive@perl.apache.org Received: (qmail 91963 invoked by uid 500); 8 Jan 2013 14:05:40 -0000 Mailing-List: contact modperl-help@perl.apache.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: List-Post: List-Id: Delivered-To: mailing list modperl@perl.apache.org Received: (qmail 91956 invoked by uid 99); 8 Jan 2013 14:05:40 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 Jan 2013 14:05:40 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [209.68.5.17] (HELO relay03.pair.com) (209.68.5.17) by apache.org (qpsmtpd/0.29) with SMTP; Tue, 08 Jan 2013 14:05:32 +0000 Received: (qmail 89456 invoked by uid 0); 8 Jan 2013 14:05:10 -0000 Received: from 209.6.49.245 (HELO ?192.168.10.2?) (209.6.49.245) by relay03.pair.com with SMTP; 8 Jan 2013 14:05:10 -0000 X-pair-Authenticated: 209.197.49.245 Subject: Re: [mp2] mod_perl 2.0.7 Segmentation fault in OutputFilter(?) From: David Booth To: Petra Zier Cc: modperl@perl.apache.org In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Date: Tue, 08 Jan 2013 09:05:08 -0500 Message-ID: <1357653908.28278.16518.camel@dbooth-laptop> Mime-Version: 1.0 X-Mailer: Evolution 2.28.3 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org I'm not a modperl expert, but I do know that modperl does some unusual things with threads and memory, to try to make separate threads act more like separate processes, i.e., to treat separate threads as though they had separate memory spaces. However, I think it may depend on which worker model you use, and I don't know what are the rules for worker MPM. With that backdrop, given that the segmentation fault is happening every 2-4 hours, from a general debugging perspective I would suspect that: (a) some resource is somehow being exhausted; or (b) something is getting corrupted because of threads being reused, maybe in an unexpected way. Sorry I don't have any more specific suggestions, but maybe someone else will. David On Tue, 2013-01-08 at 11:52 +0000, Petra Zier wrote: > Hi, > I am using httpd 2.2.23 (worker MPM), perl 5.14.2, mod-perl-2.0.7 on a > 64bit-CentOS-6.3 system and received every 2-4 hours a Segmentation > fault (maybe from the output filter). :-( > > My apache config looks like: > > Order Allow,Deny > Allow from all > RewriteEngine On > #many rules here > RewriteRule ^prb/internal/ - [L] > #IF %{REQUEST_URI} == "/" then use /xxx.html > RewriteRule ^$ http://127.0.0.1:80/xxx.html [P,QSA,L] > #ELSE > RewriteRule ^(.*)$ http://127.0.0.1:80/$1 [P,QSA,L] > > SetHandler modperl > PerlResponseHandler PrB::myHandler > PerlOutputFilterHandler PrB::OutputFilter > > > Here are three backtraces of apaches coredumps: > Does anyone has any idea what might be the reason for this? What can be done to fix this? > > #0 Perl_sv_magic (my_perl=0x57f1ec0, sv=0x10110700000001, obj=0x0, how=126, name=0x0, namlen=-1) at sv.c:5253 > #1 0x00007fd6b5fa7acd in modperl_filter_mg_set (filter=0x7fd690004980) at modperl_filter.c:358 > #2 modperl_run_filter (filter=0x7fd690004980) at modperl_filter.c:500 > #3 0x00007fd6b5fa83a5 in modperl_output_filter_handler (f=0x7fd6900090a0, bb=0x7fd690009c68) at modperl_filter.c:907 > #4 0x00007fd6b828a83f in ap_proxy_http_process_response (p=0x7fd6900028f8, r=, backend=0x2632ce0, origin=0x2640f50, > conf=0x4d71310, server_portstr=0x7fd6a97faae0 "") at mod_proxy_http.c:1835 > #5 0x00007fd6b828bc89 in proxy_http_handler (r=0x7fd690002970, worker=, conf=0x7fd6a97fab68, > url=0x7fd690009240 "/xxx.pdf", > proxyname=0x7fd6900008e8 "\270I", proxyport=3920) at mod_proxy_http.c:2036 > #6 0x00007fd6b8493e82 in proxy_run_scheme_handler (r=0x7fd690002970, worker=0x2562770, conf=0x4d71310, > url=0x7fd690008fe6 "http://127.0.0.1:80/xxx.pdf", > proxyhost=0x0, proxyport=0) at mod_proxy.c:2415 > #7 0x00007fd6b8498587 in proxy_handler (r=0x7fd690002970) at mod_proxy.c:1031 > #8 0x000000000043ca20 in ap_run_handler (r=0x7fd690002970) at config.c:160 > #9 ap_invoke_handler (r=0x7fd690002970) at config.c:378 > #10 0x0000000000449770 in ap_process_request (r=0x7fd690002970) at http_request.c:282 > #11 0x0000000000446240 in ap_process_http_connection (c=0x7fd6ac004c58) at http_core.c:190 > #12 0x0000000000442348 in ap_run_process_connection (c=0x7fd6ac004c58, csd=0x7fd6ac004a40) at connection.c:43 > #13 ap_process_connection (c=0x7fd6ac004c58, csd=0x7fd6ac004a40) at connection.c:190 > #14 0x0000000000451f84 in process_socket (thd=0x2562f80, dummy=) at worker.c:545 > #15 worker_thread (thd=0x2562f80, dummy=) at worker.c:895 > #16 0x00007fd6bc311851 in start_thread () from /lib64/libpthread.so.0 > #17 0x00007fd6bc05f11d in clone () from /lib64/libc.so.6 > > ==================== > #0 modperl_filter_f_cleanup (data=) at modperl_filter.c:287 > #1 0x00007fd6bc97fd2e in run_cleanups (pool=0x7fd660017de8) at memory/unix/apr_pools.c:2352 > #2 apr_pool_destroy (pool=0x7fd660017de8) at memory/unix/apr_pools.c:814 > #3 0x000000000044616d in ap_process_http_connection (c=0x7fd698011358) at http_core.c:199 > #4 0x0000000000442348 in ap_run_process_connection (c=0x7fd698011358, csd=0x7fd698011140) at connection.c:43 > #5 ap_process_connection (c=0x7fd698011358, csd=0x7fd698011140) at connection.c:190 > #6 0x0000000000451f84 in process_socket (thd=0x256e0f0, dummy=) at worker.c:545 > #7 worker_thread (thd=0x256e0f0, dummy=) at worker.c:895 > #8 0x00007fd6bc311851 in start_thread () from /lib64/libpthread.so.0 > #9 0x00007fd6bc05f11d in clone () from /lib64/libc.so.6 > > ============= > #0 S_more_sv (my_perl=0x57f1ec0, i=) at sv.c:249 > #1 Perl_newSViv (my_perl=0x57f1ec0, i=) at sv.c:8627 > #2 0x00007fd6b39863e9 in XS_Time__HiRes_gettimeofday (my_perl=0x57f1ec0, cv=) at HiRes.xs:1043 > #3 0x00007fd6b5cafa70 in Perl_pp_entersub (my_perl=0x57f1ec0) at pp_hot.c:3046 > #4 0x00007fd6b5cae6a6 in Perl_runops_standard (my_perl=0x57f1ec0) at run.c:41 > #5 0x00007fd6b5c433b4 in Perl_call_sv (my_perl=0x57f1ec0, sv=0x596eee0, flags=10) at perl.c:2647 > #6 0x00007fd6b5fa03a5 in modperl_callback (my_perl=0x57f1ec0, handler=0x3e22450, p=0x7fd650006918, r=0x7fd650006990, s=0x3e20240, > args=0x7fd658107f70) at modperl_callback.c:101 > #7 0x00007fd6b5fa7b06 in modperl_run_filter (filter=0x7fd68c016350) at modperl_filter.c:513 > #8 0x00007fd6b5fa83a5 in modperl_output_filter_handler (f=0x7fd65000c5e8, bb=0x7fd674024460) at modperl_filter.c:907 > #9 0x00007fd6b828a83f in ap_proxy_http_process_response (p=0x7fd650006918, r=, backend=0x2582830, origin=0x259aaf0, > conf=0x4b47c88, server_portstr=0x7fd6a83f8ae0 "") at mod_proxy_http.c:1835 > #10 0x00007fd6b828bc89 in proxy_http_handler (r=0x7fd650006990, worker=, conf=0x7fd6a83f8b68, > url=0x7fd65000c718 "/xxx.gif", proxyname=0x7fd6500008e8 "\230q\001\254\326\177", proxyport=43760) > at mod_proxy_http.c:2036 > #11 0x00007fd6b8493e82 in proxy_run_scheme_handler (r=0x7fd650006990, worker=0x2562770, conf=0x4b47c88, > url=0x7fd65000c566 "http://127.0.0.1:80/xxx.gif", proxyhost=0x0, proxyport=0) at mod_proxy.c:2415 > #12 0x00007fd6b8498587 in proxy_handler (r=0x7fd650006990) at mod_proxy.c:1031 > #13 0x000000000043ca20 in ap_run_handler (r=0x7fd650006990) at config.c:160 > #14 ap_invoke_handler (r=0x7fd650006990) at config.c:378 > #15 0x0000000000449770 in ap_process_request (r=0x7fd650006990) at http_request.c:282 > #16 0x0000000000446240 in ap_process_http_connection (c=0x7fd6ac017438) at http_core.c:190 > #17 0x0000000000442348 in ap_run_process_connection (c=0x7fd6ac017438, csd=0x7fd6ac017220) at connection.c:43 > #18 ap_process_connection (c=0x7fd6ac017438, csd=0x7fd6ac017220) at connection.c:190 > #19 0x0000000000451f84 in process_socket (thd=0x2562fe0, dummy=) at worker.c:545 > #20 worker_thread (thd=0x2562fe0, dummy=) at worker.c:895 > #21 0x00007fd6bc311851 in start_thread () from /lib64/libpthread.so.0 > #22 0x00007fd6bc05f11d in clone () from /lib64/libc.so.6 > > > perl -V says: > Summary of my perl5 (revision 5 version 14 subversion 2) configuration: > > Platform: > osname=linux, osvers=2.6.32-279.5.2.el6.x86_64, archname=x86_64-linux-thread-multi > uname='linux vwww-dev05.ws.prb.de 2.6.32-279.5.2.el6.x86_64 #1 smp fri aug 24 01:07:11 utc 2012 x86_64 x86_64 x86_64 gnulinux ' > config_args='-Dcc=cc -Doptimize=-O3 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Dprefix=/prb/opt64/perl-5.14.2 -Duselargefiles=y -Dbincompat5005=undef -Dinstallusrbinperl=undef -Dd_suidsafe=undef -Dcf_email=nobody@example.com -Dperladmin=nobody@example.com -Duseshrplib -Dusethreads -de' > hint=recommended, useposix=true, d_sigaction=define > useithreads=define, usemultiplicity=define > useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef > use64bitint=define, use64bitall=define, uselongdouble=undef > usemymalloc=n, bincompat5005=undef > Compiler: > cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', > optimize='-O3 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic', > cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' > ccversion='', gccversion='4.4.6 20120305 (Red Hat 4.4.6-4)', gccosandvers='' > intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 > d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 > ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 > alignbytes=8, prototype=define > Linker and Libraries: > ld='cc', ldflags =' -fstack-protector -L/usr/local/lib' > libpth=/usr/local/lib /lib/../lib64 /usr/lib/../lib64 /lib /usr/lib /lib64 /usr/lib64 /usr/local/lib64 > libs=-lnsl -lgdbm -ldl -lm -lcrypt -lutil -lpthread -lc > perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc > libc=/lib/libc-2.12.so, so=so, useshrplib=true, libperl=libperl.so > gnulibc_version='2.12' > Dynamic Linking: > dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/prb/opt64/perl-5.14.2/lib/5.14.2/x86_64-linux-thread-multi/CORE' > cccdlflags='-fPIC', lddlflags='-shared -O3 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -L/usr/local/lib' > > > Characteristics of this binary (from libperl): > Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV > PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP > PERL_PRESERVE_IVUV USE_64_BIT_ALL USE_64_BIT_INT > USE_ITHREADS USE_LARGE_FILES USE_PERLIO USE_PERL_ATOF > USE_REENTRANT_API > Built under linux > Compiled at Sep 14 2012 07:38:30 > @INC: > /prb/opt64/perl-5.14.2/lib/site_perl/5.14.2/x86_64-linux-thread-multi > /prb/opt64/perl-5.14.2/lib/site_perl/5.14.2 > /prb/opt64/perl-5.14.2/lib/5.14.2/x86_64-linux-thread-multi > /prb/opt64/perl-5.14.2/lib/5.14.2 > . > > > Many thanx for any hints! > Petra > > > -- David Booth, Ph.D. http://dbooth.org/ Opinions expressed herein are those of the author and do not necessarily reflect those of his employer.