httpd-modules-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <oh...@cox.net>
Subject Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use Apache module that uses shared library?
Date Thu, 01 Jan 1970 00:00:00 GMT

---- Joe Lewis <joe@joe-lewis.com> wrote: 
> On 6/22/12 6:16 AM, ohaya@cox.net wrote:
> > ---- ohaya@cox.net wrote:
> >> ---- Rainer Jung<rainer.jung@kippdata.de>  wrote:
> >>> On 22.06.2012 06:10, Joe Lewis wrote:
> >>>> On 6/21/12 10:02 PM, ohaya@cox.net wrote:
> >>>>> ---- Joe Lewis<joe@joe-lewis.com>   wrote:
> >>>>>> On 6/21/12 9:39 PM, ohaya@cox.net wrote:
> >>>>>>> ---- ohaya@cox.net wrote:
> >>>>>>>> ---- ohaya@cox.net wrote:
> >>>>>>>>> ---- Joe Lewis<joe@joe-lewis.com>    wrote:
> >>>>>>>>>> On 6/21/12 7:32 PM, ohaya@cox.net wrote:
> >>>>>>>>>>> ---- ohaya@cox.net wrote:
> >>>>>>>>>>>> ---- Joe Lewis<joe@joe-lewis.com>
    wrote:
> >>>>>>>>>>>>> On 6/21/12 6:46 PM, ohaya@cox.net
wrote:
> >>>>>>>>>>>>>> ---- Joe Lewis<joe@joe-lewis.com>
     wrote:
> >>>>>>>>>>>>>>> On 6/21/12 5:49 PM, ohaya@cox.net
wrote:
> >>>>>>>>>>>>>>>> ---- ohaya@cox.net wrote:
> >>>>>>>>>>>>>>>>> ---- Sorin Manolache<sorinm@gmail.com>
      wrote:
> >>>>>>>>>>>>>>>>>> And I forgot
to say: run gdb in some sort of environment
> >>>>>>>>>>>>>>>>>> where you see
> >>>>>>>>>>>>>>>>>> your current
source code line and a couple of surrounding
> >>>>>>>>>>>>>>>>>> lines. You
> >>>>>>>>>>>>>>>>>> could achieve
this with the "list" command, but I prefer
> >>>>>>>>>>>>>>>>>> running gdb
in
> >>>>>>>>>>>>>>>>>> emacs and let
emacs do the nice listing of source code in
> >>>>>>>>>>>>>>>>>> a different
panel.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> S
> >>>>>>>>>>>>>>>>> Here's the function
from my source.  It's the original
> >>>>>>>>>>>>>>>>> from mod_headers.c,
plus my printf:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> static int header_post_config(apr_pool_t
*pconf,
> >>>>>>>>>>>>>>>>> apr_pool_t *plog,
> >>>>>>>>>>>>>>>>>                
                    apr_pool_t *ptemp,
> >>>>>>>>>>>>>>>>> server_rec *s)
> >>>>>>>>>>>>>>>>> {
> >>>>>>>>>>>>>>>>>           printf("In
header_post_config\n");
> >>>>>>>>>>>>>>>>>           header_ssl_lookup
=
> >>>>>>>>>>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>>>>>>>>>>>           return
OK;
> >>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Jim
> >>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I was able to get the
segfault to go away.  Here's what I
> >>>>>>>>>>>>>>>> had to do:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> - Created /etc/ld.so.conf.d/my.conf,
and added the
> >>>>>>>>>>>>>>>> directory where my libobaccess.so
was
> >>>>>>>>>>>>>>>> - Run 'ldconfig' to
activate.
> >>>>>>>>>>>>>>>> - In the apxs command,
DON'T include the -L and -l arguments
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> After that, Apache appears
to start ok, without segfault :)!!
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks for all of the
great help, esp. the suggestion about
> >>>>>>>>>>>>>>>> checking "ldconfig -p".
 I still don't understand why, but
> >>>>>>>>>>>>>>>> I'm just glad that I
can get past this piece so now I can
> >>>>>>>>>>>>>>>> debug my module :)...
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Later,
> >>>>>>>>>>>>>>>> Jim
> >>>>>>>>>>>>>>> I'm just glad this list
is as good as it is!
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> FYI, the ldconfig is the
dynamic linker control, and those
> >>>>>>>>>>>>>>> /etc/ld.so.conf.d files
provide additional search
> >>>>>>>>>>>>>>> directories for the
> >>>>>>>>>>>>>>> linker to check in when
loading a library.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Joe
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> http://www.silverhawk.net/
> >>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Sorry to report, but my earlier
report was a "false positive"
> >>>>>>>>>>>>>> :)...
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I forgot that the mod_headers.c
that I was doing the earlier
> >>>>>>>>>>>>>> testing with had all references
to the libobaccess.so removed
> >>>>>>>>>>>>>> :(!!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> So, I'm still stuck with basically
the same problem, now,
> >>>>>>>>>>>>>> working with my "full" code,
with the calls in it:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> - If I compile with -L and -l,
Apache segfaults when it starts
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> - If I compile without -L and
-l, then I get "undefined
> >>>>>>>>>>>>>> symbol" errors when I try to
start Apache, e.g.:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> [root@apachemodule build-mod_headers]#
> >>>>>>>>>>>>>> /apps/httpd2222/bin/apachectl
-k start -X
> >>>>>>>>>>>>>> httpd: Syntax error on line
84 of
> >>>>>>>>>>>>>> /apps/httpd2222/conf/httpd.conf:
Cannot load
> >>>>>>>>>>>>>> /apps/httpd2222/modules/mod_headers.so
into server:
> >>>>>>>>>>>>>> /apps/httpd2222/modules/mod_headers.so:
undefined symbol:
> >>>>>>>>>>>>>> ObResource_isProtected
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> That "ObResource_isProtected"
should be a symbol in
> >>>>>>>>>>>>>> libobaccess.so, and in fact,
if I do "nm --dynamic", I get:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> [root@apachemodule build-mod_headers]#
nm --dynamic
> >>>>>>>>>>>>>> /apps/netpoint/lib64/libobaccess.so
| grep
> >>>>>>>>>>>>>> "ObResource_isProtected"
> >>>>>>>>>>>>>> 00000000000a6d80 T ObResource_isProtected
> >>>>>>>>>>>>>> [root@apachemodule build-mod_headers]#
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I'm *assuming* that the reason
for the "undefined symbol"
> >>>>>>>>>>>>>> error is that libobaccess.so
is actually not being loaded,
> >>>>>>>>>>>>>> but then when I try to load
libobaccess.so, either via -L and
> >>>>>>>>>>>>>> -l in the apxs, or using LoadFile
in httpd.conf, I get the
> >>>>>>>>>>>>>> segfault (same gdb info, BTW).
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Catch-22?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Sorry for the false alarm :(!!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Jim
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> Not a catch-22.  The -L and -l specify
linker options when
> >>>>>>>>>>>>> assembling
> >>>>>>>>>>>>> the code.  The ldconfig is a run-time
thing.  If you are
> >>>>>>>>>>>>> getting the
> >>>>>>>>>>>>> stderr messages, you are making
it all the way into your
> >>>>>>>>>>>>> library.  I'd
> >>>>>>>>>>>>> suggest commenting out the following
line and see if you get
> >>>>>>>>>>>>> farther :
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> That should tell you if the problem
is the ssl_var_lookup.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Joe
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> http://www.silverhawk.net/
> >>>>>>>>>>>> Hi,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks for the suggestion.  I just tried
what you suggested,
> >>>>>>>>>>>> and got a segfault when I started Apache
with the modified module.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Jim
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>> Hi,
> >>>>>>>>>>>
> >>>>>>>>>>> As a reminder, here's the gdb with the library
loaded:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> (gdb) b header_post_config
> >>>>>>>>>>> Function "header_post_config" not defined.
> >>>>>>>>>>> Make breakpoint pending on future shared
library load? (y or [n]) y
> >>>>>>>>>>> Breakpoint 1 (header_post_config) pending.
> >>>>>>>>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> >>>>>>>>>>> Starting program: /apps/httpd2222/bin/httpd
-d /apps/httpd2222/
> >>>>>>>>>>> -f /apps/httpd2222/conf/httpd.conf
> >>>>>>>>>>> [Thread debugging using libthread_db enabled]
> >>>>>>>>>>> [New Thread 182897610272 (LWP 11317)]
> >>>>>>>>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c,
line 1121.
> >>>>>>>>>>> Pending breakpoint "header_post_config"
resolved
> >>>>>>>>>>> mod_headers-jl V0.09 - start calling OAM
API
> >>>>>>>>>>> In register_hooks
> >>>>>>>>>>> In create_headers_dir_config
> >>>>>>>>>>> In create_headers_dir_config
> >>>>>>>>>>> In header_cmd
> >>>>>>>>>>> In header_inout_cmd
> >>>>>>>>>>> In parse_format_tag
> >>>>>>>>>>> In parse_misc_string
> >>>>>>>>>>> In create_headers_dir_config
> >>>>>>>>>>> In header_cmd
> >>>>>>>>>>> In header_inout_cmd
> >>>>>>>>>>> In parse_format_tag
> >>>>>>>>>>> In parse_misc_string
> >>>>>>>>>>> [Switching to Thread 182897610272 (LWP 11317)]
> >>>>>>>>>>>
> >>>>>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138,
plog=0x5a52c8,
> >>>>>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> >>>>>>>>>>> 1121        printf("In header_post_config\n");
> >>>>>>>>>>> (gdb) n
> >>>>>>>>>>> 1120    {
> >>>>>>>>>>> (gdb) n
> >>>>>>>>>>> 1121        printf("In header_post_config\n");
> >>>>>>>>>>> (gdb) n
> >>>>>>>>>>> In header_post_config
> >>>>>>>>>>> 1122        header_ssl_lookup =
> >>>>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>>>>> (gdb) n
> >>>>>>>>>>> 1124    }
> >>>>>>>>>>> (gdb) n
> >>>>>>>>>>> 0x00000000004360c7 in ap_run_post_config
(pconf=0x573138,
> >>>>>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8)
at config.c:91
> >>>>>>>>>>> 91      AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> >>>>>>>>>>> (gdb) n
> >>>>>>>>>>>
> >>>>>>>>>>> Program received signal SIGSEGV, Segmentation
fault.
> >>>>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from
/lib64/libcrypto.so.4
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> So, it's actually blowing up in "BN_num_bits()"
in
> >>>>>>>>>>> /lib64/libcrypto.so.4?
> >>>>>>>>>>>
> >>>>>>>>>>> Jim
> >>>>>>>>>> When you see the :
> >>>>>>>>>>
> >>>>>>>>>> Program received signal SIGSEGV, Segmentation
fault.
> >>>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> in gdb, type "bt" and hit enter.  It should
show the back trace
> >>>>>>>>>> and how
> >>>>>>>>>> you got to the BN_num_bits() function.
> >>>>>>>>>>
> >>>>>>>>>> Joe
> >>>>>>>>>> --
> >>>>>>>>>> http://www.silverhawk.net/
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> Here's the bt full:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> In header_inout_cmd
> >>>>>>>>> In parse_format_tag
> >>>>>>>>> In parse_misc_string
> >>>>>>>>> [Switching to Thread 182897610272 (LWP 6676)]
> >>>>>>>>>
> >>>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138,
plog=0x5a52c8,
> >>>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> >>>>>>>>> 1121        printf("In header_post_config\n");
> >>>>>>>>> (gdb) n
> >>>>>>>>> 1120    {
> >>>>>>>>> (gdb) n
> >>>>>>>>> 1121        printf("In header_post_config\n");
> >>>>>>>>> (gdb) n
> >>>>>>>>> In header_post_config
> >>>>>>>>> 1122        header_ssl_lookup =
> >>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>>> (gdb) n
> >>>>>>>>> 1124    }
> >>>>>>>>> (gdb) n
> >>>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
> >>>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> >>>>>>>>> 91      AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> >>>>>>>>> (gdb) n
> >>>>>>>>>
> >>>>>>>>> Program received signal SIGSEGV, Segmentation fault.
> >>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>>>>>> (gdb) bt full
> >>>>>>>>> #0  0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #1  0x0000003518da8f4e in X509_ATTRIBUTE_create
() from
> >>>>>>>>> /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #2  0x0000003518dadea2 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #3  0x0000003518dadf79 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #4  0x0000003518dae0e1 in ASN1_item_ex_i2d () from
> >>>>>>>>> /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #5  0x0000003518dae5f2 in ASN1_template_i2d () from
> >>>>>>>>> /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #6  0x0000003518dae28e in ASN1_item_ex_i2d () from
> >>>>>>>>> /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #7  0x0000003518dae3c6 in ASN1_item_i2d () from
/lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #8  0x0000002a987d9d3a in ssl_pphrase_Handle (s=0x59d3a8,
> >>>>>>>>> p=0x5a72d8) at ssl_engine_pphrase.c:505
> >>>>>>>>>            mc = (SSLModConfigRec *) 0x571738
> >>>>>>>>>            sc = (SSLSrvConfigRec *) 0x668c38
> >>>>>>>>>            pServ = (server_rec *) 0x65fa48
> >>>>>>>>>            cpVHostID = 0x60add0 "www.example.com:443"
> >>>>>>>>>            szPath =
> >>>>>>>>> "/apps/httpd2222/conf/certs/apache1.whatever.com.key\000\177\000\000\000Ü׿\177\000\000\000à׿\177",
> >>>>>>>>> '\0'<rep                                    
           eats 15
> >>>>>>>>> times>,
> >>>>>>>>> "\001\000\000\000°å¿\177\000\000\000;\000\000\000+\000\000\000\020Û¿\177",
> >>>>>>>>> '\0'<repeats 35 times>, "à\224k",
> >>>>>>>>> '                                              
 ---Type<return>
> >>>>>>>>> to continue, or q<return>    to quit---
> >>>>>>>>> \0'<repeats 13 times>,
> >>>>>>>>> "øé¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
> >>>>>>>>> '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°å¿\177",
> >>>>>>>>> '\0'<repeats 11 times>, "\020Û¿\177\000\000\000+\000\000\000;",
> >>>>>>>>> '\0'<repeats 35 times>...
> >>>>>>>>>            pPrivateKey = (EVP_PKEY *) 0x6ba670
> >>>>>>>>>            asn1 = Variable "asn1" is not available.
> >>>>>>>>> (gdb)
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Not really sure what to make of that though :(...
> >>>>>>>>>
> >>>>>>>>> Jim
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> Ok, something interesting.  I noticed that the servername
in the
> >>>>>>>> gdb output was example.whatever.com, so I changed ServerName
in
> >>>>>>>> extras/httpd-ssl.conf to "apache1.whatever.com".  Apache
still
> >>>>>>>> segfaulted after that.
> >>>>>>>>
> >>>>>>>> I noticed that igdb was showing my router IP address
for
> >>>>>>>> "mod_unique_id" (whatever that is), so I thought it
was because I
> >>>>>>>> didn't have the hostname in /etc/hosts.  So, I added
> >>>>>>>> "apache1.whatever.com" to /etc/hosts, and also turned
Apache
> >>>>>>>> LogLevel to debug, and now I get a different error:
> >>>>>>>>
> >>>>>>>> [root@apachemodule dev]# /apps/httpd2222/bin/apachectl
-k start -X
> >>>>>>>> mod_headers-jl V0.09 - start calling OAM API
> >>>>>>>> In register_hooks
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In header_cmd
> >>>>>>>> In header_inout_cmd
> >>>>>>>> In parse_format_tag
> >>>>>>>> In parse_misc_string
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In header_cmd
> >>>>>>>> In header_inout_cmd
> >>>>>>>> In parse_format_tag
> >>>>>>>> In parse_misc_string
> >>>>>>>> In header_post_config
> >>>>>>>> *** glibc detected *** corrupted double-linked list:
> >>>>>>>> 0x00000000006b9710 ***
> >>>>>>>> /apps/httpd2222/bin/apachectl: line 78:  7599
> >>>>>>>> Aborted                 $HTTPD $ARGV
> >>>>>>>> [root@apachemodule dev]#
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Here's the Apache error_log:
> >>>>>>>>
> >>>>>>>> [Thu Jun 21 23:24:11 2012] [info] mod_unique_id: using
ip addr
> >>>>>>>> 72.215.225.9
> >>>>>>>> [Thu Jun 21 23:24:12 2012] [info] Init: Seeding PRNG
with 144 bytes
> >>>>>>>> of entropy
> >>>>>>>> [Thu Jun 21 23:24:12 2012] [info] Loading certificate&
   private
> >>>>>>>> key of SSL-aware server
> >>>>>>>> [Thu Jun 21 23:29:51 2012] [info] mod_unique_id: using
ip addr
> >>>>>>>> 72.215.225.9
> >>>>>>>> [Thu Jun 21 23:29:52 2012] [info] Init: Seeding PRNG
with 144 bytes
> >>>>>>>> of entropy
> >>>>>>>> [Thu Jun 21 23:29:52 2012] [info] Loading certificate&
   private
> >>>>>>>> key of SSL-aware server
> >>>>>>>> [Thu Jun 21 23:29:56 2012] [info] mod_unique_id: using
ip addr
> >>>>>>>> 72.215.225.9
> >>>>>>>> [Thu Jun 21 23:29:57 2012] [info] Init: Seeding PRNG
with 144 bytes
> >>>>>>>> of entropy
> >>>>>>>> [Thu Jun 21 23:29:57 2012] [info] Loading certificate&
   private
> >>>>>>>> key of SSL-aware server
> >>>>>>>> [root@apachemodule dev]#
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> and here's the gdb backtrace:
> >>>>>>>>
> >>>>>>>> (gdb) b header_post_config
> >>>>>>>> Function "header_post_config" not defined.
> >>>>>>>> Make breakpoint pending on future shared library load?
(y or [n]) y
> >>>>>>>> Breakpoint 1 (header_post_config) pending.
> >>>>>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> >>>>>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/
-f
> >>>>>>>> /apps/httpd2222/conf/httpd.conf
> >>>>>>>> [Thread debugging using libthread_db enabled]
> >>>>>>>> [New Thread 182897610272 (LWP 7644)]
> >>>>>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line
1121.
> >>>>>>>> Pending breakpoint "header_post_config" resolved
> >>>>>>>> mod_headers-jl V0.09 - start calling OAM API
> >>>>>>>> In register_hooks
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In header_cmd
> >>>>>>>> In header_inout_cmd
> >>>>>>>> In parse_format_tag
> >>>>>>>> In parse_misc_string
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In header_cmd
> >>>>>>>> In header_inout_cmd
> >>>>>>>> In parse_format_tag
> >>>>>>>> In parse_misc_string
> >>>>>>>> [Switching to Thread 182897610272 (LWP 7644)]
> >>>>>>>>
> >>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
> >>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> >>>>>>>> 1121        printf("In header_post_config\n");
> >>>>>>>> (gdb) n
> >>>>>>>> 1120    {
> >>>>>>>> (gdb) n
> >>>>>>>> 1121        printf("In header_post_config\n");
> >>>>>>>> (gdb) n
> >>>>>>>> In header_post_config
> >>>>>>>> 1122        header_ssl_lookup =
> >>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>> (gdb) n
> >>>>>>>> 1124    }
> >>>>>>>> (gdb) n
> >>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
> >>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> >>>>>>>> 91      AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> >>>>>>>> (gdb) n
> >>>>>>>> *** glibc detected *** corrupted double-linked list:
> >>>>>>>> 0x00000000006b9710 ***
> >>>>>>>>
> >>>>>>>> Program received signal SIGABRT, Aborted.
> >>>>>>>> 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
> >>>>>>>> (gdb) bt full
> >>>>>>>> #0  0x000000351432e26d in raise () from /lib64/tls/libc.so.6
> >>>>>>>> No symbol table info available.
> >>>>>>>> #1  0x000000351432fa6e in abort () from /lib64/tls/libc.so.6
> >>>>>>>> No symbol table info available.
> >>>>>>>> #2  0x0000003514363641 in __libc_message () from /lib64/tls/libc.so.6
> >>>>>>>> No symbol table info available.
> >>>>>>>> #3  0x0000003514369512 in _int_free () from /lib64/tls/libc.so.6
> >>>>>>>> No symbol table info available.
> >>>>>>>> #4  0x0000003514369846 in free () from /lib64/tls/libc.so.6
> >>>>>>>> No symbol table info available.
> >>>>>>>> #5  0x0000002a9790b6ba in R_free () from
> >>>>>>>> /apps/netpoint/lib64/libobaccess.so
> >>>>>>>> No symbol table info available.
> >>>>>>>> #6  0x0000002a9792bc41 in X509_CINF_free () from
> >>>>>>>> /apps/netpoint/lib64/libobaccess.so
> >>>>>>>> No symbol table info available.
> >>>>>>>> #7  0x0000002a9790ed98 in X509_free () from
> >>>>>>>> /apps/netpoint/lib64/libobaccess.so
> >>>>>>>> No symbol table info available.
> >>>>>>>> #8  0x0000002a987d97b6 in ssl_pphrase_Handle (s=0x59d3a8,
> >>>>>>>> p=0x5a72d8) at ssl_engine_pphrase.c:243
> >>>>>>>>            mc = (SSLModConfigRec *) 0x571738
> >>>>>>>>            sc = (SSLSrvConfigRec *) 0x668c38
> >>>>>>>>            pServ = (server_rec *) 0x65fa48
> >>>>>>>>            cpVHostID = 0x60add0 "apache1.whatever.com:443"
> >>>>>>>>            szPath =
> >>>>>>>> "/apps/httpd2222/conf/certs/apache1.whatever.com.crt\000\177\000\000\000Ü׿\177\000\000\000à׿\177",
> >>>>>>>> '\0'<repeats 15 times>,
> >>>>>>>> "\001\000\000\000°å¿\177\000\000\000;\000\000\000+\000\000\000\020Û¿\177",
> >>>>>>>> '\0'<repeats 35 times>, "à\224k", '\0'<repeats
13 times>,
> >>>>>>>> "øé¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
> >>>>>>>> '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°å¿\177",
> >>>>>>>> '\0'<repeats 11 times>, "\020Û¿\177\000\000\000+\000\000\000;",
> >>>>>>>> '\0'<repeats 35 times>...
> >>>>>>>>            pPrivateKey = Variable "pPrivateKey" is not
available.
> >>>>>>>> (gdb)
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Jim
> >>>>>>>>
> >>>>>>> I guess the thing that I'm puzzled about is why it goes
from
> >>>>>>> ssl_pphrase_Handle (which is part of Apache code?) to X509_free
()
> >>>>>>> in the libobaccess.so?  Apache shouldn't even be aware of
the
> >>>>>>> libobaccess stuff yet (since I think that Apache is still
trying to
> >>>>>>> initialize SSL processing)?
> >>>>>>>
> >>>>>>> Jim
> >>>>>> It shouldn't unless you've declared a shutdown hook or a clean
up hook
> >>>>>> of sorts. The looks of that stack trace look strikingly like
an
> >>>>>> exception handler firing off.  What are all of the hooks you
have
> >>>>>> running?
> >>>>>>
> >>>>>> Joe
> >>>>>> --
> >>>>>> http://www.silverhawk.net/
> >>>>>>
> >>>>> Hi,
> >>>>>
> >>>>> I'm basically using the original mod_headers.c as a starter code,
and
> >>>>> stuck some stuff in there.  Haven't done anything with the hooks
etc.
> >>>>> yet, so whatever was there is there:
> >>>>>
> >>>>> static void register_hooks(apr_pool_t *p)
> >>>>> {
> >>>>>       printf("mod_headers-jl V0.09 - start calling OAM API\n");
> >>>>>       printf("In register_hooks\n");
> >>>>>       ap_register_output_filter("FIXUP_HEADERS_OUT",
> >>>>> ap_headers_output_filter,
> >>>>>                                 NULL, AP_FTYPE_CONTENT_SET);
> >>>>>       ap_register_output_filter("FIXUP_HEADERS_ERR",
> >>>>> ap_headers_error_filter,
> >>>>>                                 NULL, AP_FTYPE_CONTENT_SET);
> >>>>>       ap_hook_pre_config(header_pre_config,NULL,NULL,APR_HOOK_MIDDLE);
> >>>>>       ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
> >>>>>       ap_hook_insert_filter(ap_headers_insert_output_filter, NULL,
> >>>>> NULL, APR_HOOK_LAST);
> >>>>>       ap_hook_insert_error_filter(ap_headers_insert_error_filter,
> >>>>>                                   NULL, NULL, APR_HOOK_LAST);
> >>>>>       ap_hook_fixups(ap_headers_fixup, NULL, NULL, APR_HOOK_LAST);
> >>>>>       ap_hook_post_read_request(ap_headers_early, NULL, NULL,
> >>>>> APR_HOOK_FIRST);
> >>>>> }
> >>>>>
> >>>>> module AP_MODULE_DECLARE_DATA headers_module =
> >>>>> {
> >>>>>       STANDARD20_MODULE_STUFF,
> >>>>>       create_headers_dir_config,  /* dir config creater */
> >>>>>       merge_headers_config,       /* dir merger --- default is to
> >>>>> override */
> >>>>>       NULL,                       /* server config */
> >>>>>       NULL,                       /* merge server configs */
> >>>>>       headers_cmds,               /* command apr_table_t */
> >>>>>       register_hooks              /* register hooks */
> >>>>> };
> >>>>>
> >>>>> I guess that there is a:
> >>>>>
> >>>>> ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
> >>>>>
> >>>>> in that register_hooks(), but it was there from the original code.
> >>>>>
> >>>>> BTW, I was checking, and it looks like there's an X509_free() function
> >>>>> in both openssl and in libobaccess.so (used 'nm --dynamic<file>),
so
> >>>>> I'm wondering if things are confused, and maybe Apache code expects
to
> >>>>> call the openssl X509_free, but somehow ends up calling the
> >>>>> X509_free() in libobaccess.so?
> >>>>>
> >>>>> What would cause something like that?  If two .so files have the
same
> >>>>> function, how to avoid conflicts?
> >>>>>
> >>>>> Jim
> >>>> Jim, you may have hit that on the head.  I'm not sure of the load order
> >>>> and looking up symbols, but it could very well change things.  Try
> >>>> re-ordering or adding a LoadFile before the SSL module and see if that
> >>>> changes things.
> >>> Look up order usually is load order. So if mod_ssl loads first and loads
> >>> its dependency libcrypto, the symbol will always be found there even if
> >>> the implementation in libobaccess.so is needed - and vice versa.
> >>>
> >>> On Solaris there is a -Bdirect linker flag which changes runtime search
> >>> order so that each module would find a needed symbol first inits direct
> >>> dependencies. Unfortunately AFAIK notihing similar exists for Linux.
> >>>
> >>> You would need to find ssl libs and libobaccess with compatible symbols.
> >>> Best would be a libobaccess which is dynamically linked against a
> >>> compatible version of libcrypto instead of - as it seems - being
> >>> statically linked against it.
> >>>
> >>> Regards,
> >>>
> >>> Rainer
> >>>
> >>
> >> Hi Rainer,
> >>
> >> Here's exactly what I found:
> >>
> >> [root@apachemodule ~]# ldd /apps/netpoint/lib64/libobaccess.so
> >>          libnsl.so.1 =>  /lib64/libnsl.so.1 (0x0000002a95ab8000)
> >>          libdl.so.2 =>  /lib64/libdl.so.2 (0x0000002a95bd0000)
> >>          libpthread.so.0 =>  /lib64/tls/libpthread.so.0 (0x0000002a95cd3000)
> >>          libstdc++.so.6 =>  /lib64/libstdc++.so.6 (0x0000002a95de9000)
> >>          libm.so.6 =>  /lib64/tls/libm.so.6 (0x0000002a95fd9000)
> >>          libgcc_s.so.1 =>  /lib64/libgcc_s.so.1 (0x0000002a9615f000)
> >>          libc.so.6 =>  /lib64/tls/libc.so.6 (0x0000002a9626d000)
> >>          /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
> >> [root@apachemodule ~]#
> >> [root@apachemodule ~]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep
"X509_free"
> >> 00000000002d8d60 T X509_free
> >>
> >>
> >> [root@apachemodule ~]# ldd /lib64/libssl.so.0.9.7a
> >>          libgssapi_krb5.so.2 =>  /usr/lib64/libgssapi_krb5.so.2 (0x0000003518700000)
> >>          libkrb5.so.3 =>  /usr/lib64/libkrb5.so.3 (0x0000003519000000)
> >>          libcom_err.so.2 =>  /lib64/libcom_err.so.2 (0x0000003518500000)
> >>          libk5crypto.so.3 =>  /usr/lib64/libk5crypto.so.3 (0x0000003518900000)
> >>          libcrypto.so.4 =>  /lib64/libcrypto.so.4 (0x0000003518d00000)
> >>          libdl.so.2 =>  /lib64/libdl.so.2 (0x0000003514800000)
> >>          libz.so.1 =>  /usr/lib64/libz.so.1 (0x0000003514c00000)
> >>          libc.so.6 =>  /lib64/tls/libc.so.6 (0x0000003514300000)
> >>          libresolv.so.2 =>  /lib64/libresolv.so.2 (0x0000003518300000)
> >>          /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
> >> [root@apachemodule ~]#
> >> [root@apachemodule ~]# nm --dynamic /lib64/libssl.so.0.9.7a | grep "X509_free"
> >>                   U X509_free
> >>
> >>
> >> [root@apachemodule ~]# ldd /lib64/libcrypto.so.0.9.7a
> >>          libgssapi_krb5.so.2 =>  /usr/lib64/libgssapi_krb5.so.2 (0x0000003518700000)
> >>          libkrb5.so.3 =>  /usr/lib64/libkrb5.so.3 (0x0000003519000000)
> >>          libcom_err.so.2 =>  /lib64/libcom_err.so.2 (0x0000003518500000)
> >>          libk5crypto.so.3 =>  /usr/lib64/libk5crypto.so.3 (0x0000003518900000)
> >>          libdl.so.2 =>  /lib64/libdl.so.2 (0x0000003514800000)
> >>          libz.so.1 =>  /usr/lib64/libz.so.1 (0x0000003514c00000)
> >>          libc.so.6 =>  /lib64/tls/libc.so.6 (0x0000003514300000)
> >>          libresolv.so.2 =>  /lib64/libresolv.so.2 (0x0000003518300000)
> >>          /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
> >> [root@apachemodule ~]#
> >> [root@apachemodule ~]# nm --dynamic /lib64/libcrypto.so.0.9.7a | grep "X509_free"
> >> 0000003518da9860 T X509_free
> >>
> >>
> >> So:
> >>
> >> - liboaccess.so doesn't REFERENCE libcrypto, but it HAS an X509_free() function
in it.
> >> - libssl.so.REFERENCES libcrypto, AND it HAS an X509_free function in it.
> >> - libcrypto also has an X509_free function in it.
> >>
> >>  From your msg above, are you saying that this situation can't be resolved by
trying to change load order?
> >>
> >> And the only way is what you said in your last paragraph (find another libobccess.so)?
> >>
> >> Jim
> >>
> >>
> > Hi,
> >
> > Also, BTW, the call to X509_free is apparently because in ssl_engine_pphrase.c,
which it seems is part of mod_ssl, it has:
> >
> >              /*
> >               * Free the X509 structure
> >               */
> >              X509_free(pX509Cert);
> >
> > but that's calling the X509_free() in libobaccess.so, rather than the X509_free()
in libcrypt.so?
> >
> > Jim
> 
> The way I read it, the load order can help.  Did you try changing 
> locations of the LoadFile for libobaccess.so to before or after the 
> LoadModule for the SSL engine?
> 
> Joe
> --
> http://www.silverhawk.net/


Hi,

No, not yet.  I was waiting for more feedback from here :).  But I will do it now and post
back.  Right now the LoadFile for libobaccess.so is in front of (earlier in httpd.conf) the
Loadmodule.

Jim

Mime
View raw message