Return-Path: Delivered-To: apmail-httpd-users-archive@www.apache.org Received: (qmail 79311 invoked from network); 30 Apr 2007 23:36:24 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 30 Apr 2007 23:36:24 -0000 Received: (qmail 72133 invoked by uid 500); 30 Apr 2007 23:36:19 -0000 Delivered-To: apmail-httpd-users-archive@httpd.apache.org Received: (qmail 71814 invoked by uid 500); 30 Apr 2007 23:36:17 -0000 Mailing-List: contact users-help@httpd.apache.org; run by ezmlm Precedence: bulk Reply-To: users@httpd.apache.org list-help: list-unsubscribe: List-Post: List-Id: Delivered-To: mailing list users@httpd.apache.org Received: (qmail 71803 invoked by uid 99); 30 Apr 2007 23:36:17 -0000 Received: from herse.apache.org (HELO herse.apache.org) (140.211.11.133) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 30 Apr 2007 16:36:17 -0700 X-ASF-Spam-Status: No, hits=2.0 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (herse.apache.org: domain of yanmercier@gmail.com designates 64.233.166.183 as permitted sender) Received: from [64.233.166.183] (HELO py-out-1112.google.com) (64.233.166.183) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 30 Apr 2007 16:36:07 -0700 Received: by py-out-1112.google.com with SMTP id u77so1975583pyb for ; Mon, 30 Apr 2007 16:35:46 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=HLQSxvmZbBqzwBhAOO15U4b6+bJcAgBjja6TvbXR+QA5qh4PP/uAVzNHflx7+Xkz+mjy0k0+zch9RtGD80yi9LlHHj9hA7eICegzhW5xqSBby5wMHq6PNwmcQiNZ2KCsi5WD/t8owHt1OCPqzDGOzp4fj274KVg7YR/I21Y9uyQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=cUjVR7CgK2jro6YVFBpwbLuqimso68wagmqd53euvMCPmUJI6MHcq5nKdg1Ynbo6qBzyZLRFknBsrQCb+Y4u9DP/3hPCA0ZWHammUy2+5ggcYrjHgHO27xkLtH8KJxy9WYlpO1uA5mEIo761DUaquHVbUtwLRwTcsnZFo+1CIRE= Received: by 10.64.199.8 with SMTP id w8mr12941296qbf.1177976144128; Mon, 30 Apr 2007 16:35:44 -0700 (PDT) Received: by 10.65.200.19 with HTTP; Mon, 30 Apr 2007 16:35:44 -0700 (PDT) Message-ID: <76f910780704301635neef881bq1e5927bd230c3143@mail.gmail.com> Date: Mon, 30 Apr 2007 19:35:44 -0400 From: "Yannick Mercier" To: users@httpd.apache.org Cc: sctemme@apache.org In-Reply-To: <76f910780704231001w7d05072byae84289ac41348de@mail.gmail.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_64097_12673297.1177976144049" References: <76f910780704230812x328ba78fq80d2d808f4242104@mail.gmail.com> <78FA2697-E2B0-4AF7-BF77-406EBC7B42B0@apache.org> <76f910780704231001w7d05072byae84289ac41348de@mail.gmail.com> X-Virus-Checked: Checked by ClamAV on apache.org Subject: Re: [users@httpd] any luck using shared memory for mod_ssl SSLSessionCache ? ------=_Part_64097_12673297.1177976144049 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline I found out how to backtrace the core files from httpd crash due to SSLSessionCache, and here is the output of the backtrace from Sun Studio Debugger Any help with this backtrace to find out if this is due to a misconfiguration, or a bug in the software would be appreciated: dbx /opt/apache/bin/httpd /var/core/core.httpd.2372 For information about new features see `help changes' To remove this message, put `dbxenv suppress_startup_message 7.5' in your .dbxrc Reading httpd core file header read successfully Reading ld.so.1 Reading libaprutil-1.so.0 Reading libldap-2.3.so.0 Reading liblber-2.3.so.0 Reading libexpat.so.0 Reading libapr-1.so.0 Reading libuuid.so.1 Reading libsendfile.so.1 Reading librt.so.1 Reading libsocket.so.1 Reading libnsl.so.1 Reading libpthread.so.1 Reading libdl.so.1 Reading libthread.so.1 Reading libc.so.1 Reading libucb.so.1 Reading libresolv.so.2 Reading libelf.so.1 Reading libgen.so.1 Reading libaio.so.1 Reading libmp.so.2 Reading libc_psr.so.1 Reading nss_files.so.1 Reading mod_authz_host.so Reading mod_auth_basic.so Reading mod_deflate.so Reading libz.so.1 Reading mod_ldap.so Reading mod_log_config.so Reading mod_log_forensic.so Reading mod_logio.so Reading mod_env.so Reading mod_headers.so Reading mod_setenvif.so Reading mod_ssl.so Reading mod_mime.so Reading mod_status.so Reading mod_info.so Reading mod_dir.so Reading mod_alias.so Reading mod_rewrite.so Reading mod_perl.so Reading libperl.so Reading libmalloc.so.1 Reading libm.so.1 Reading libsec.so.1 Reading mod_authnz_ldap.so Reading mod_evasive20.so Reading Embperl.so t@1 (l@1) terminated by signal BUS (invalid address alignment) 0xffffffff7945b664: shmcb_lookup_session_id+0x0264: ld [%i5 + 12], %g1 (dbx) where -h current thread: t@1 =>[1] shmcb_lookup_session_id(0x10024d470, 0xffffffff7fff8760, 0xffffffff7fff8740, 0x10038e5fb, 0x10, 0xffffffff7c127152), at 0xffffffff7945b664 [2] shmcb_retrieve_session(0x10024d470, 0xffffffff7c100008, 0x60, 0x270e2, 0x10, 0xa), at 0xffffffff79459134 [3] ssl_scache_shmcb_retrieve(0x10024d470, 0x10038e5fb, 0x10, 0x0, 0x26e0d4, 0xffffffff796c5a18), at 0xffffffff79457990 [4] ssl_callback_GetSessionCacheEntry(0x1003a04b0, 0x10038e5fb, 0x10, 0xffffffff7fff8a4c, 0x0, 0x10), at 0xffffffff7944ad98 [5] ssl_get_prev_session(0x1, 0x10038e5fb, 0x10, 0x1, 0x10036b7c0, 0xaa), at 0xffffffff794743f4 [6] get_client_hello(0x1003a04b0, 0x10038e5e0, 0x32, 0x10, 0xffffffff796c5a18, 0x12), at 0xffffffff7945fac4 [7] ssl2_accept(0x1003a04b0, 0x1, 0x0, 0x2070, 0x2010, 0x2030), at 0xffffffff7945e4a8 [8] ssl23_get_client_hello(0x1003a04b0, 0x6000000, 0x0, 0x10039f360, 0xffffffff796c5a18, 0x1), at 0xffffffff79466f98 [9] ssl23_accept(0x1003a04b0, 0x0, 0x2210, 0x6000, 0xffffffff796c5a18, 0x4000), at 0xffffffff7946612c [10] ssl_io_filter_connect(0x10037a7f0, 0x100379f58, 0x0, 0x10039cd70, 0x1003479b8, 0x10037a6e8), at 0xffffffff7944601c [11] ssl_io_filter_input(0x100385588, 0x10039e548, 0x1, 0x0, 0x0, 0x1001dca48), at 0xffffffff794467cc [12] ap_rgetline_core(0x10039ce38, 0x2000, 0xffffffff7fff91e0, 0x10039ce08, 0x10039e550, 0x10039e548), at 0x100029074 [13] ap_read_request(0x100379f58, 0x10039ce38, 0x100385588, 0x10024d470, 0x10039ce08, 0x1), at 0x10002a6f0 [14] ap_process_http_connection(0x100379f58, 0x100379f58, 0x1, 0x100176758, 0x126b14, 0x1), at 0x10004fc68 [15] ap_process_connection(0x100379f58, 0x1001a2f88, 0xbee0, 0x1003488a0, 0x100182638, 0x4), at 0x100049a68 [16] child_main(0x100379f58, 0x1001826a0, 0x10017e0cc, 0x100176758, 0x10017df80, 0x100182758), at 0x100059fcc [17] make_child(0x7400, 0x1, 0x0, 0x11c594, 0x100176758, 0x1000594a0), at 0x10005a330 [18] startup_children(0x5, 0x100, 0x7628, 0xff000000, 0x10017e0c0, 0x100176758), at 0x10005a40c [19] ap_mpm_run(0xffffffff7bf00008, 0x5, 0x100197b70, 0x10017df88, 0x100176758, 0x1001826e0), at 0x10005a6d0 [20] main(0x7400, 0x0, 0x10017de68, 0x100197b70, 0x1001905d8, 0x100176758), at 0x10001de04 (dbx) Thank you Yannick On 4/23/07, Yannick Mercier wrote: > > Yep Ive got some core files in /var/core, one for every time > a child crashes > > my SSLMutex is set to pthreads > ive tried different settings for SSLMutex in combination > with different settings for SSLSessionCache and I concluded > that problems happen when SSLSessionCache is set to shm > independently of SSLMutex settings > > there is a debugger installed with Sun Studio at /opt/SUNWspro/bin/dbx > but Im not familiar with it dont know how to run a backtrace on the core > file > > I could upload the core file somewhere > > > On 4/23/07, Sander Temme wrote: > > > > > > On Apr 23, 2007, at 8:12 AM, Yannick Mercier wrote: > > > > > [Mon Mar 19 08:45:28 2007] [notice] child pid 27827 exit signal Bus > > > error (10), possible coredump in /opt/apache > > > > If the httpd child can write to /opt/apache, you're likely to find a > > core file there. We'd be hugely interested in a backtrace of that. > > That'd give us the state of the system when the crash occurred, and > > would give us insight into what went wrong. > > > > If you don't find that core file, manipulate the core location to be > > a directory to which the httpd children have write access. See the > > CoreDumpDirectory directive and coreadm(1M). > > > > What's your SSLMutex? Have you tried different settings for that? > > > > > Anyone can help fixing this ? any suggestions ? I built apache with > > > Sun Studio 11 under Solaris 8 with mod_authnz_ldap/openldap in 64bit > > > with these flags to the compiler -fast -xarch=v9b -xcode=pic32 , I > > > then compiled mod_perl and mod_evasive using apxs > > > > I'm not sure how you take backtraces with the debugger that comes > > with Sun Studio. I have it installed on a VMWare image, but before I > > can try figure this out I have a lot of bears to cook, so if you, or > > someone else, beats me to it that'd be better for all. > > > > S. > > > > -- > > Sander Temme > > sctemme@apache.org > > PGP FP: 51B4 8727 466A 0BC3 69F4 B7B8 B2BE BC40 1529 24AF > > > > > > > > > > > ------=_Part_64097_12673297.1177976144049 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline
I found out how to backtrace the core files from httpd crash due to SSLSessionCache, and here is the output of the backtrace from Sun Studio Debugger
Any help with this backtrace to find out if this is due to a misconfiguration, or a bug in the software would be appreciated:
 
dbx /opt/apache/bin/httpd /var/core/core.httpd.2372
For information about new features see `help changes'
To remove this message, put `dbxenv suppress_startup_message 7.5' in your .dbxrc
Reading httpd
core file header read successfully
Reading ld.so.1
Reading libaprutil-1.so.0
Reading libldap-2.3.so.0
Reading liblber-2.3.so.0
Reading libexpat.so.0
Reading libapr-1.so.0
Reading libuuid.so.1
Reading libsendfile.so.1
Reading librt.so.1
Reading libsocket.so.1
Reading libnsl.so.1
Reading libpthread.so.1
Reading libdl.so.1
Reading libthread.so.1
Reading libc.so.1
Reading libucb.so.1
Reading libresolv.so.2
Reading libelf.so.1
Reading libgen.so.1
Reading libaio.so.1
Reading libmp.so.2
Reading libc_psr.so.1
Reading nss_files.so.1
Reading mod_authz_host.so
Reading mod_auth_basic.so
Reading mod_deflate.so
Reading libz.so.1
Reading mod_ldap.so
Reading mod_log_config.so
Reading mod_log_forensic.so
Reading mod_logio.so
Reading mod_env.so
Reading mod_headers.so
Reading mod_setenvif.so
Reading mod_ssl.so
Reading mod_mime.so
Reading mod_status.so
Reading mod_info.so
Reading mod_dir.so
Reading mod_alias.so
Reading mod_rewrite.so
Reading mod_perl.so
Reading libperl.so
Reading libmalloc.so.1
Reading libm.so.1
Reading libsec.so.1
Reading mod_authnz_ldap.so
Reading mod_evasive20.so
Reading Embperl.so
t@1 (l@1) terminated by signal BUS (invalid address alignment)
0xffffffff7945b664: shmcb_lookup_session_id+0x0264:     ld       [%i5 + 12], %g1
(dbx) where -h                                                              
current thread: t@1
=>[1] shmcb_lookup_session_id(0x10024d470, 0xffffffff7fff8760, 0xffffffff7fff8740, 0x10038e5fb, 0x10, 0xffffffff7c127152), at 0xffffffff7945b664
  [2] shmcb_retrieve_session(0x10024d470, 0xffffffff7c100008, 0x60, 0x270e2, 0x10, 0xa), at 0xffffffff79459134
  [3] ssl_scache_shmcb_retrieve(0x10024d470, 0x10038e5fb, 0x10, 0x0, 0x26e0d4, 0xffffffff796c5a18), at 0xffffffff79457990
  [4] ssl_callback_GetSessionCacheEntry(0x1003a04b0, 0x10038e5fb, 0x10, 0xffffffff7fff8a4c, 0x0, 0x10), at 0xffffffff7944ad98
  [5] ssl_get_prev_session(0x1, 0x10038e5fb, 0x10, 0x1, 0x10036b7c0, 0xaa), at 0xffffffff794743f4
  [6] get_client_hello(0x1003a04b0, 0x10038e5e0, 0x32, 0x10, 0xffffffff796c5a18, 0x12), at 0xffffffff7945fac4
  [7] ssl2_accept(0x1003a04b0, 0x1, 0x0, 0x2070, 0x2010, 0x2030), at 0xffffffff7945e4a8
  [8] ssl23_get_client_hello(0x1003a04b0, 0x6000000, 0x0, 0x10039f360, 0xffffffff796c5a18, 0x1), at 0xffffffff79466f98
  [9] ssl23_accept(0x1003a04b0, 0x0, 0x2210, 0x6000, 0xffffffff796c5a18, 0x4000), at 0xffffffff7946612c
  [10] ssl_io_filter_connect(0x10037a7f0, 0x100379f58, 0x0, 0x10039cd70, 0x1003479b8, 0x10037a6e8), at 0xffffffff7944601c
  [11] ssl_io_filter_input(0x100385588, 0x10039e548, 0x1, 0x0, 0x0, 0x1001dca48), at 0xffffffff794467cc
  [12] ap_rgetline_core(0x10039ce38, 0x2000, 0xffffffff7fff91e0, 0x10039ce08, 0x10039e550, 0x10039e548), at 0x100029074
  [13] ap_read_request(0x100379f58, 0x10039ce38, 0x100385588, 0x10024d470, 0x10039ce08, 0x1), at 0x10002a6f0
  [14] ap_process_http_connection(0x100379f58, 0x100379f58, 0x1, 0x100176758, 0x126b14, 0x1), at 0x10004fc68
  [15] ap_process_connection(0x100379f58, 0x1001a2f88, 0xbee0, 0x1003488a0, 0x100182638, 0x4), at 0x100049a68
  [16] child_main(0x100379f58, 0x1001826a0, 0x10017e0cc, 0x100176758, 0x10017df80, 0x100182758), at 0x100059fcc
  [17] make_child(0x7400, 0x1, 0x0, 0x11c594, 0x100176758, 0x1000594a0), at 0x10005a330
  [18] startup_children(0x5, 0x100, 0x7628, 0xff000000, 0x10017e0c0, 0x100176758), at 0x10005a40c
  [19] ap_mpm_run(0xffffffff7bf00008, 0x5, 0x100197b70, 0x10017df88, 0x100176758, 0x1001826e0), at 0x10005a6d0
  [20] main(0x7400, 0x0, 0x10017de68, 0x100197b70, 0x1001905d8, 0x100176758), at 0x10001de04
(dbx)
 
 
 
Thank you
Yannick

 
On 4/23/07, Yannick Mercier <yanmercier@gmail.com> wrote:
Yep Ive got some core files in /var/core, one for every time
a child crashes
 
my SSLMutex is set to pthreads
ive tried different settings for SSLMutex in combination
with different settings for SSLSessionCache and I concluded
that problems happen when SSLSessionCache is set to shm
independently of SSLMutex settings
 
there is a debugger installed with Sun Studio at /opt/SUNWspro/bin/dbx
but Im not familiar with it dont know how to run a backtrace on the core file
 
I could upload the core file somewhere

 
On 4/23/07, Sander Temme <sctemme@apache.org > wrote:

On Apr 23, 2007, at 8:12 AM, Yannick Mercier wrote:

> [Mon Mar 19 08:45:28 2007] [notice] child pid 27827 exit signal Bus
> error (10), possible coredump in /opt/apache

If the httpd child can write to /opt/apache, you're likely to find a
core file there.  We'd be hugely interested in a backtrace of that.
That'd give us the state of the system when the crash occurred, and
would give us insight into what went wrong.

If you don't find that core file, manipulate the core location to be
a directory to which the httpd children have write access. See the
CoreDumpDirectory directive and coreadm(1M).

What's your SSLMutex? Have you tried different settings for that?

> Anyone can help fixing this ? any suggestions ? I built apache with
> Sun Studio 11 under Solaris 8 with mod_authnz_ldap/openldap in 64bit
> with these flags to the compiler -fast -xarch=v9b -xcode=pic32 , I
> then compiled mod_perl and mod_evasive using apxs

I'm not sure how you take backtraces with the debugger that comes
with Sun Studio.  I have it installed on a VMWare image, but before I
can try figure this out I have a lot of bears to cook, so if you, or
someone else, beats me to it that'd be better for all.

S.

--
Sander Temme
sctemme@apache.org
PGP FP: 51B4 8727 466A 0BC3 69F4  B7B8 B2BE BC40 1529 24AF






------=_Part_64097_12673297.1177976144049--