My problem appears to be resolved, though I do not know why.

I ended up rebuilding apr and apr-utils using Gentoo's portage system, after that, my apache server no longer segment faulted.

Here's what was updated:
[I] dev-libs/apr
     Available versions:  (1) 1.3.9!t 1.4.2!t
        {doc older-kernels-compatibility +urandom}
     Installed versions:  1.4.2(1)!t(07:01:50 AM 01/16/2011)(urandom -doc -older-kernels-compatibility)
     Homepage:            http://apr.apache.org/
     Description:         Apache Portable Runtime Library

[I] dev-libs/apr-util
     Available versions:  (1) 1.3.9!t ~1.3.9-r1!t 1.3.10!t
        {berkdb doc freetds gdbm ldap mysql odbc postgres sqlite sqlite3}
     Installed versions:  1.3.10(1)!t(07:09:20 AM 01/16/2011)(berkdb gdbm ldap mysql postgres -doc -freetds -odbc -sqlite -sqlite3)
     Homepage:            http://apr.apache.org/
     Description:         Apache Portable Runtime Utility Library

What is new is that the apr-util was new.  The apr was simply a recompile.  I'll consider this matter closed and resolved, unless there is something more someone would like me to try in the test instances I created to further shed light on what was causing the problem.

On 1/15/2011 10:20 AM, John L. Poole wrote:
Hi Sander,

Dismayed, I needed to take a break from the fact that Apache on my ARM (SheevaPlug) rapidly dies with a segmentation fault provided little trail with which to debug. 

Background:
Here are the bugs I filed in Gentoo which simply further document the problem I experienced.

https://bugs.gentoo.org/show_bug.cgi?id=347910
www-servers/apache-2.2.16 builds then Segmentation faults at start up
I'm the only commentator

https://bugs.gentoo.org/show_bug.cgi?id=347908
www-apache/libapreq2-2.12 fails to build: configure: error: Bad apache2 binary (/usr/sbin/apache2)
I'm basically the only commentator (fixed an ommission)

I then posted to this mailing list on 12/13/2010.

The last suggestion was Sander's, so here is what I did.  I'm not quite sure how I should execute the "on top" part of your suggestion

Current install.

Apache:
I downloaded  httpd-2.2.16.tar.bz2 and created a working directory at /usr/local/src/httpd-2.2.16
I created an output working area: /var/tmp/apache/httpd

Within /usr/local/src/httpd-2.2.16  I did the following:

    plug httpd-2.2.16 # CFLAGS="-DDEBUG -O0"; export CFLAGS
    plug httpd-2.2.16 # ./configure --prefix=/var/tmp/apache/httpd --enable-maintainer-mode
    plug httpd-2.2.16 # make
    plug httpd-2.2.16 # make install
    plug httpd-2.2.16 # cd /var/tmp/apache/httpd/bin

    plug httpd-2.2.16 # gdb httpd
    GNU gdb (Gentoo 7.0.1 p1) 7.0.1
    Copyright (C) 2009 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
    and "show warranty" for details.
    This GDB was configured as "armv5tel-softfloat-linux-gnueabi".
    For bug reporting instructions, please see:
    <http://bugs.gentoo.org/>...
    Reading symbols from /usr/local/src/httpd-2.2.16/httpd...(no debugging symbols found)...done.
    warning: not using untrusted file "/usr/local/src/httpd-2.2.16/.gdbinit"
    (gdb) run -X
    Starting program: /usr/local/src/httpd-2.2.16/httpd -X
    [Thread debugging using libthread_db enabled]

    Program received signal SIGSEGV, Segmentation fault.
    0x402600e4 in apr_palloc () from /usr/lib/libapr-1.so.0
    (gdb)) bt
    #0  0x402600e4 in apr_palloc () from /usr/lib/libapr-1.so.0
    #1  0x4025a988 in apr_array_make () from /usr/lib/libapr-1.so.0
    #2  0x400ddb54 in apr_hook_sort_register () from /usr/lib/libaprutil-1.so.0
    #3  0x00051f74 in ap_hook_create_connection ()
    #4  0x0003f838 in register_hooks ()
    #5  0x000485e4 in ap_register_hooks ()
    #6  0x00048974 in ap_add_module ()
    #7  0x00048d64 in ap_setup_prelinked_modules ()
    #8  0x00028be0 in main ()
    (gdb) quit
    ...
   
    plug httpd-2.2.16 # gdb httpd
    GNU gdb (Gentoo 7.0.1 p1) 7.0.1
    Copyright (C) 2009 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
    and "show warranty" for details.
    This GDB was configured as "armv5tel-softfloat-linux-gnueabi".
    For bug reporting instructions, please see:
    <http://bugs.gentoo.org/>...
    Reading symbols from /usr/local/src/httpd-2.2.16/httpd...(no debugging symbols found)...done.
    warning: not using untrusted file "/usr/local/src/httpd-2.2.16/.gdbinit"
    (gdb) run
    Starting program: /usr/local/src/httpd-2.2.16/httpd
    [Thread debugging using libthread_db enabled]

    Program received signal SIGSEGV, Segmentation fault.
    0x402600e4 in apr_palloc () from /usr/lib/libapr-1.so.0
    (gdb) bt
    #0  0x402600e4 in apr_palloc () from /usr/lib/libapr-1.so.0
    #1  0x4025a988 in apr_array_make () from /usr/lib/libapr-1.so.0
    #2  0x400ddb54 in apr_hook_sort_register () from /usr/lib/libaprutil-1.so.0
    #3  0x00051f74 in ap_hook_create_connection ()
    #4  0x0003f838 in register_hooks ()
    #5  0x000485e4 in ap_register_hooks ()
    #6  0x00048974 in ap_add_module ()
    #7  0x00048d64 in ap_setup_prelinked_modules ()
    #8  0x00028be0 in main ()
    (gdb)

APR:
I have apr-1.4.2  downloaded, but am not sure how to install it so that my custom install of apache at /var/tmp/apache utilizes it.

plug apr-1.4.2 #make clean
plug apr-1.4.2 # ...
plug apr-1.4.2 # echo $CFLAGS
-DDEBUG -O0
plug apr-1.4.2 # ./configure --enable-maintainer-mode
plug apr-1.4.2 # make

The output from ./configure, make,make install, and gdb are posted at:

http://snipt.org/wknpm

For convenience, here is the back trace:

#0  0x402600e4 in apr_palloc () from /usr/lib/libapr-1.so.0
#1  0x4025a988 in apr_array_make () from /usr/lib/libapr-1.so.0
#2  0x400ddb54 in apr_hook_sort_register () from /usr/lib/libaprutil-1.so.0
#3  0x00051f74 in ap_hook_create_connection ()
#4  0x0003f838 in register_hooks ()
#5  0x000485e4 in ap_register_hooks ()
#6  0x00048974 in ap_add_module ()
#7  0x00048d64 in ap_setup_prelinked_modules ()
#8  0x00028be0 in main ()

Please let me know if there is any other compile or debug step I should take to create something helpful to identify the problem. 

Thank you for you help,

John



On 12/22/2010 7:57 AM, Sander Temme wrote:
On Dec 22, 2010, at 6:20 AM, John L. Poole wrote:

Am I missing something, as the output above seems the same as when I did not have debugging
How are you making Apache use your new library?  I suggest compiling your own Apache on top of your own APR/APR-Util, and debugging against that. 

In the system build, something is clearly happening that horribly confuses the debugger. 

S.


--
Email Rider

John Laurence Poole
P.O. Box 6566
Napa CA 94581-6566
707-812-1323 office


--
Email Rider

John Laurence Poole
P.O. Box 6566
Napa CA 94581-6566
707-812-1323 office