httpd-modules-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rainer Jung <rainer.j...@kippdata.de>
Subject Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use Apache module that uses shared library?
Date Fri, 22 Jun 2012 13:29:08 GMT
On 22.06.2012 14:16, ohaya@cox.net wrote:
>
> ---- ohaya@cox.net wrote:
>>
>> ---- Rainer Jung <rainer.jung@kippdata.de> wrote:
>>> On 22.06.2012 06:10, Joe Lewis wrote:
>>>> On 6/21/12 10:02 PM, ohaya@cox.net wrote:
>>>>> ---- Joe Lewis<joe@joe-lewis.com>  wrote:
>>>>>> On 6/21/12 9:39 PM, ohaya@cox.net wrote:
>>>>>>> ---- ohaya@cox.net wrote:
>>>>>>>> ---- ohaya@cox.net wrote:
>>>>>>>>> ---- Joe Lewis<joe@joe-lewis.com>   wrote:
>>>>>>>>>> On 6/21/12 7:32 PM, ohaya@cox.net wrote:
>>>>>>>>>>> ---- ohaya@cox.net wrote:
>>>>>>>>>>>> ---- Joe Lewis<joe@joe-lewis.com> 
  wrote:
>>>>>>>>>>>>> On 6/21/12 6:46 PM, ohaya@cox.net wrote:
>>>>>>>>>>>>>> ---- Joe Lewis<joe@joe-lewis.com>
    wrote:
>>>>>>>>>>>>>>> On 6/21/12 5:49 PM, ohaya@cox.net
wrote:
>>>>>>>>>>>>>>>> ---- ohaya@cox.net wrote:
>>>>>>>>>>>>>>>>> ---- Sorin Manolache<sorinm@gmail.com>
     wrote:
>>>>>>>>>>>>>>>>>> And I forgot to say:
run gdb in some sort of environment
>>>>>>>>>>>>>>>>>> where you see
>>>>>>>>>>>>>>>>>> your current source
code line and a couple of surrounding
>>>>>>>>>>>>>>>>>> lines. You
>>>>>>>>>>>>>>>>>> could achieve this
with the "list" command, but I prefer
>>>>>>>>>>>>>>>>>> running gdb in
>>>>>>>>>>>>>>>>>> emacs and let emacs
do the nice listing of source code in
>>>>>>>>>>>>>>>>>> a different panel.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> S
>>>>>>>>>>>>>>>>> Here's the function from
my source.  It's the original
>>>>>>>>>>>>>>>>> from mod_headers.c, plus
my printf:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> static int header_post_config(apr_pool_t
*pconf,
>>>>>>>>>>>>>>>>> apr_pool_t *plog,
>>>>>>>>>>>>>>>>>                     
               apr_pool_t *ptemp,
>>>>>>>>>>>>>>>>> server_rec *s)
>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>           printf("In
header_post_config\n");
>>>>>>>>>>>>>>>>>           header_ssl_lookup
=
>>>>>>>>>>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>>>>>>>>>>>           return OK;
>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Jim
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I was able to get the segfault
to go away.  Here's what I
>>>>>>>>>>>>>>>> had to do:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> - Created /etc/ld.so.conf.d/my.conf,
and added the
>>>>>>>>>>>>>>>> directory where my libobaccess.so
was
>>>>>>>>>>>>>>>> - Run 'ldconfig' to activate.
>>>>>>>>>>>>>>>> - In the apxs command, DON'T
include the -L and -l arguments
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> After that, Apache appears
to start ok, without segfault :)!!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks for all of the great
help, esp. the suggestion about
>>>>>>>>>>>>>>>> checking "ldconfig -p". 
I still don't understand why, but
>>>>>>>>>>>>>>>> I'm just glad that I can
get past this piece so now I can
>>>>>>>>>>>>>>>> debug my module :)...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Later,
>>>>>>>>>>>>>>>> Jim
>>>>>>>>>>>>>>> I'm just glad this list is as
good as it is!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> FYI, the ldconfig is the dynamic
linker control, and those
>>>>>>>>>>>>>>> /etc/ld.so.conf.d files provide
additional search
>>>>>>>>>>>>>>> directories for the
>>>>>>>>>>>>>>> linker to check in when loading
a library.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> http://www.silverhawk.net/
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sorry to report, but my earlier report
was a "false positive"
>>>>>>>>>>>>>> :)...
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I forgot that the mod_headers.c that
I was doing the earlier
>>>>>>>>>>>>>> testing with had all references to
the libobaccess.so removed
>>>>>>>>>>>>>> :(!!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So, I'm still stuck with basically
the same problem, now,
>>>>>>>>>>>>>> working with my "full" code, with
the calls in it:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - If I compile with -L and -l, Apache
segfaults when it starts
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - If I compile without -L and -l,
then I get "undefined
>>>>>>>>>>>>>> symbol" errors when I try to start
Apache, e.g.:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [root@apachemodule build-mod_headers]#
>>>>>>>>>>>>>> /apps/httpd2222/bin/apachectl -k
start -X
>>>>>>>>>>>>>> httpd: Syntax error on line 84 of
>>>>>>>>>>>>>> /apps/httpd2222/conf/httpd.conf:
Cannot load
>>>>>>>>>>>>>> /apps/httpd2222/modules/mod_headers.so
into server:
>>>>>>>>>>>>>> /apps/httpd2222/modules/mod_headers.so:
undefined symbol:
>>>>>>>>>>>>>> ObResource_isProtected
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> That "ObResource_isProtected" should
be a symbol in
>>>>>>>>>>>>>> libobaccess.so, and in fact, if I
do "nm --dynamic", I get:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [root@apachemodule build-mod_headers]#
nm --dynamic
>>>>>>>>>>>>>> /apps/netpoint/lib64/libobaccess.so
| grep
>>>>>>>>>>>>>> "ObResource_isProtected"
>>>>>>>>>>>>>> 00000000000a6d80 T ObResource_isProtected
>>>>>>>>>>>>>> [root@apachemodule build-mod_headers]#
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm *assuming* that the reason for
the "undefined symbol"
>>>>>>>>>>>>>> error is that libobaccess.so is actually
not being loaded,
>>>>>>>>>>>>>> but then when I try to load libobaccess.so,
either via -L and
>>>>>>>>>>>>>> -l in the apxs, or using LoadFile
in httpd.conf, I get the
>>>>>>>>>>>>>> segfault (same gdb info, BTW).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Catch-22?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sorry for the false alarm :(!!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Jim
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>> Not a catch-22.  The -L and -l specify
linker options when
>>>>>>>>>>>>> assembling
>>>>>>>>>>>>> the code.  The ldconfig is a run-time
thing.  If you are
>>>>>>>>>>>>> getting the
>>>>>>>>>>>>> stderr messages, you are making it all
the way into your
>>>>>>>>>>>>> library.  I'd
>>>>>>>>>>>>> suggest commenting out the following
line and see if you get
>>>>>>>>>>>>> farther :
>>>>>>>>>>>>>
>>>>>>>>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>>>>>>>
>>>>>>>>>>>>> That should tell you if the problem is
the ssl_var_lookup.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Joe
>>>>>>>>>>>>> --
>>>>>>>>>>>>> http://www.silverhawk.net/
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for the suggestion.  I just tried
what you suggested,
>>>>>>>>>>>> and got a segfault when I started Apache
with the modified module.
>>>>>>>>>>>>
>>>>>>>>>>>> Jim
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> As a reminder, here's the gdb with the library
loaded:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> (gdb) b header_post_config
>>>>>>>>>>> Function "header_post_config" not defined.
>>>>>>>>>>> Make breakpoint pending on future shared library
load? (y or [n]) y
>>>>>>>>>>> Breakpoint 1 (header_post_config) pending.
>>>>>>>>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
>>>>>>>>>>> Starting program: /apps/httpd2222/bin/httpd -d
/apps/httpd2222/
>>>>>>>>>>> -f /apps/httpd2222/conf/httpd.conf
>>>>>>>>>>> [Thread debugging using libthread_db enabled]
>>>>>>>>>>> [New Thread 182897610272 (LWP 11317)]
>>>>>>>>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c,
line 1121.
>>>>>>>>>>> Pending breakpoint "header_post_config" resolved
>>>>>>>>>>> mod_headers-jl V0.09 - start calling OAM API
>>>>>>>>>>> In register_hooks
>>>>>>>>>>> In create_headers_dir_config
>>>>>>>>>>> In create_headers_dir_config
>>>>>>>>>>> In header_cmd
>>>>>>>>>>> In header_inout_cmd
>>>>>>>>>>> In parse_format_tag
>>>>>>>>>>> In parse_misc_string
>>>>>>>>>>> In create_headers_dir_config
>>>>>>>>>>> In header_cmd
>>>>>>>>>>> In header_inout_cmd
>>>>>>>>>>> In parse_format_tag
>>>>>>>>>>> In parse_misc_string
>>>>>>>>>>> [Switching to Thread 182897610272 (LWP 11317)]
>>>>>>>>>>>
>>>>>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138,
plog=0x5a52c8,
>>>>>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
>>>>>>>>>>> 1121        printf("In header_post_config\n");
>>>>>>>>>>> (gdb) n
>>>>>>>>>>> 1120    {
>>>>>>>>>>> (gdb) n
>>>>>>>>>>> 1121        printf("In header_post_config\n");
>>>>>>>>>>> (gdb) n
>>>>>>>>>>> In header_post_config
>>>>>>>>>>> 1122        header_ssl_lookup =
>>>>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>>>>> (gdb) n
>>>>>>>>>>> 1124    }
>>>>>>>>>>> (gdb) n
>>>>>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
>>>>>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at
config.c:91
>>>>>>>>>>> 91      AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
>>>>>>>>>>> (gdb) n
>>>>>>>>>>>
>>>>>>>>>>> Program received signal SIGSEGV, Segmentation
fault.
>>>>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> So, it's actually blowing up in "BN_num_bits()"
in
>>>>>>>>>>> /lib64/libcrypto.so.4?
>>>>>>>>>>>
>>>>>>>>>>> Jim
>>>>>>>>>> When you see the :
>>>>>>>>>>
>>>>>>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> in gdb, type "bt" and hit enter.  It should show
the back trace
>>>>>>>>>> and how
>>>>>>>>>> you got to the BN_num_bits() function.
>>>>>>>>>>
>>>>>>>>>> Joe
>>>>>>>>>> --
>>>>>>>>>> http://www.silverhawk.net/
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Here's the bt full:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> In header_inout_cmd
>>>>>>>>> In parse_format_tag
>>>>>>>>> In parse_misc_string
>>>>>>>>> [Switching to Thread 182897610272 (LWP 6676)]
>>>>>>>>>
>>>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
>>>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
>>>>>>>>> 1121        printf("In header_post_config\n");
>>>>>>>>> (gdb) n
>>>>>>>>> 1120    {
>>>>>>>>> (gdb) n
>>>>>>>>> 1121        printf("In header_post_config\n");
>>>>>>>>> (gdb) n
>>>>>>>>> In header_post_config
>>>>>>>>> 1122        header_ssl_lookup =
>>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>>> (gdb) n
>>>>>>>>> 1124    }
>>>>>>>>> (gdb) n
>>>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
>>>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
>>>>>>>>> 91      AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
>>>>>>>>> (gdb) n
>>>>>>>>>
>>>>>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>>>>> (gdb) bt full
>>>>>>>>> #0  0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #1  0x0000003518da8f4e in X509_ATTRIBUTE_create () from
>>>>>>>>> /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #2  0x0000003518dadea2 in asn1_ex_i2c () from /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #3  0x0000003518dadf79 in asn1_ex_i2c () from /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #4  0x0000003518dae0e1 in ASN1_item_ex_i2d () from
>>>>>>>>> /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #5  0x0000003518dae5f2 in ASN1_template_i2d () from
>>>>>>>>> /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #6  0x0000003518dae28e in ASN1_item_ex_i2d () from
>>>>>>>>> /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #7  0x0000003518dae3c6 in ASN1_item_i2d () from /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #8  0x0000002a987d9d3a in ssl_pphrase_Handle (s=0x59d3a8,
>>>>>>>>> p=0x5a72d8) at ssl_engine_pphrase.c:505
>>>>>>>>>            mc = (SSLModConfigRec *) 0x571738
>>>>>>>>>            sc = (SSLSrvConfigRec *) 0x668c38
>>>>>>>>>            pServ = (server_rec *) 0x65fa48
>>>>>>>>>            cpVHostID = 0x60add0 "www.example.com:443"
>>>>>>>>>            szPath =
>>>>>>>>> "/apps/httpd2222/conf/certs/apache1.whatever.com.key\000\177\000\000\000Ü׿\177\000\000\000à׿\177",
>>>>>>>>> '\0'<rep                                         
      eats 15
>>>>>>>>> times>,
>>>>>>>>> "\001\000\000\000°å¿\177\000\000\000;\000\000\000+\000\000\000\020Û¿\177",
>>>>>>>>> '\0'<repeats 35 times>, "à\224k",
>>>>>>>>> '                                                ---Type<return>
>>>>>>>>> to continue, or q<return>   to quit---
>>>>>>>>> \0'<repeats 13 times>,
>>>>>>>>> "øé¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
>>>>>>>>> '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°å¿\177",
>>>>>>>>> '\0'<repeats 11 times>, "\020Û¿\177\000\000\000+\000\000\000;",
>>>>>>>>> '\0'<repeats 35 times>...
>>>>>>>>>            pPrivateKey = (EVP_PKEY *) 0x6ba670
>>>>>>>>>            asn1 = Variable "asn1" is not available.
>>>>>>>>> (gdb)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Not really sure what to make of that though :(...
>>>>>>>>>
>>>>>>>>> Jim
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Ok, something interesting.  I noticed that the servername
in the
>>>>>>>> gdb output was example.whatever.com, so I changed ServerName
in
>>>>>>>> extras/httpd-ssl.conf to "apache1.whatever.com".  Apache
still
>>>>>>>> segfaulted after that.
>>>>>>>>
>>>>>>>> I noticed that igdb was showing my router IP address for
>>>>>>>> "mod_unique_id" (whatever that is), so I thought it was because
I
>>>>>>>> didn't have the hostname in /etc/hosts.  So, I added
>>>>>>>> "apache1.whatever.com" to /etc/hosts, and also turned Apache
>>>>>>>> LogLevel to debug, and now I get a different error:
>>>>>>>>
>>>>>>>> [root@apachemodule dev]# /apps/httpd2222/bin/apachectl -k
start -X
>>>>>>>> mod_headers-jl V0.09 - start calling OAM API
>>>>>>>> In register_hooks
>>>>>>>> In create_headers_dir_config
>>>>>>>> In create_headers_dir_config
>>>>>>>> In header_cmd
>>>>>>>> In header_inout_cmd
>>>>>>>> In parse_format_tag
>>>>>>>> In parse_misc_string
>>>>>>>> In create_headers_dir_config
>>>>>>>> In header_cmd
>>>>>>>> In header_inout_cmd
>>>>>>>> In parse_format_tag
>>>>>>>> In parse_misc_string
>>>>>>>> In header_post_config
>>>>>>>> *** glibc detected *** corrupted double-linked list:
>>>>>>>> 0x00000000006b9710 ***
>>>>>>>> /apps/httpd2222/bin/apachectl: line 78:  7599
>>>>>>>> Aborted                 $HTTPD $ARGV
>>>>>>>> [root@apachemodule dev]#
>>>>>>>>
>>>>>>>>
>>>>>>>> Here's the Apache error_log:
>>>>>>>>
>>>>>>>> [Thu Jun 21 23:24:11 2012] [info] mod_unique_id: using ip
addr
>>>>>>>> 72.215.225.9
>>>>>>>> [Thu Jun 21 23:24:12 2012] [info] Init: Seeding PRNG with
144 bytes
>>>>>>>> of entropy
>>>>>>>> [Thu Jun 21 23:24:12 2012] [info] Loading certificate&
  private
>>>>>>>> key of SSL-aware server
>>>>>>>> [Thu Jun 21 23:29:51 2012] [info] mod_unique_id: using ip
addr
>>>>>>>> 72.215.225.9
>>>>>>>> [Thu Jun 21 23:29:52 2012] [info] Init: Seeding PRNG with
144 bytes
>>>>>>>> of entropy
>>>>>>>> [Thu Jun 21 23:29:52 2012] [info] Loading certificate&
  private
>>>>>>>> key of SSL-aware server
>>>>>>>> [Thu Jun 21 23:29:56 2012] [info] mod_unique_id: using ip
addr
>>>>>>>> 72.215.225.9
>>>>>>>> [Thu Jun 21 23:29:57 2012] [info] Init: Seeding PRNG with
144 bytes
>>>>>>>> of entropy
>>>>>>>> [Thu Jun 21 23:29:57 2012] [info] Loading certificate&
  private
>>>>>>>> key of SSL-aware server
>>>>>>>> [root@apachemodule dev]#
>>>>>>>>
>>>>>>>>
>>>>>>>> and here's the gdb backtrace:
>>>>>>>>
>>>>>>>> (gdb) b header_post_config
>>>>>>>> Function "header_post_config" not defined.
>>>>>>>> Make breakpoint pending on future shared library load? (y
or [n]) y
>>>>>>>> Breakpoint 1 (header_post_config) pending.
>>>>>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
>>>>>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/
-f
>>>>>>>> /apps/httpd2222/conf/httpd.conf
>>>>>>>> [Thread debugging using libthread_db enabled]
>>>>>>>> [New Thread 182897610272 (LWP 7644)]
>>>>>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
>>>>>>>> Pending breakpoint "header_post_config" resolved
>>>>>>>> mod_headers-jl V0.09 - start calling OAM API
>>>>>>>> In register_hooks
>>>>>>>> In create_headers_dir_config
>>>>>>>> In create_headers_dir_config
>>>>>>>> In header_cmd
>>>>>>>> In header_inout_cmd
>>>>>>>> In parse_format_tag
>>>>>>>> In parse_misc_string
>>>>>>>> In create_headers_dir_config
>>>>>>>> In header_cmd
>>>>>>>> In header_inout_cmd
>>>>>>>> In parse_format_tag
>>>>>>>> In parse_misc_string
>>>>>>>> [Switching to Thread 182897610272 (LWP 7644)]
>>>>>>>>
>>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
>>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
>>>>>>>> 1121        printf("In header_post_config\n");
>>>>>>>> (gdb) n
>>>>>>>> 1120    {
>>>>>>>> (gdb) n
>>>>>>>> 1121        printf("In header_post_config\n");
>>>>>>>> (gdb) n
>>>>>>>> In header_post_config
>>>>>>>> 1122        header_ssl_lookup =
>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>> (gdb) n
>>>>>>>> 1124    }
>>>>>>>> (gdb) n
>>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
>>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
>>>>>>>> 91      AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
>>>>>>>> (gdb) n
>>>>>>>> *** glibc detected *** corrupted double-linked list:
>>>>>>>> 0x00000000006b9710 ***
>>>>>>>>
>>>>>>>> Program received signal SIGABRT, Aborted.
>>>>>>>> 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
>>>>>>>> (gdb) bt full
>>>>>>>> #0  0x000000351432e26d in raise () from /lib64/tls/libc.so.6
>>>>>>>> No symbol table info available.
>>>>>>>> #1  0x000000351432fa6e in abort () from /lib64/tls/libc.so.6
>>>>>>>> No symbol table info available.
>>>>>>>> #2  0x0000003514363641 in __libc_message () from /lib64/tls/libc.so.6
>>>>>>>> No symbol table info available.
>>>>>>>> #3  0x0000003514369512 in _int_free () from /lib64/tls/libc.so.6
>>>>>>>> No symbol table info available.
>>>>>>>> #4  0x0000003514369846 in free () from /lib64/tls/libc.so.6
>>>>>>>> No symbol table info available.
>>>>>>>> #5  0x0000002a9790b6ba in R_free () from
>>>>>>>> /apps/netpoint/lib64/libobaccess.so
>>>>>>>> No symbol table info available.
>>>>>>>> #6  0x0000002a9792bc41 in X509_CINF_free () from
>>>>>>>> /apps/netpoint/lib64/libobaccess.so
>>>>>>>> No symbol table info available.
>>>>>>>> #7  0x0000002a9790ed98 in X509_free () from
>>>>>>>> /apps/netpoint/lib64/libobaccess.so
>>>>>>>> No symbol table info available.
>>>>>>>> #8  0x0000002a987d97b6 in ssl_pphrase_Handle (s=0x59d3a8,
>>>>>>>> p=0x5a72d8) at ssl_engine_pphrase.c:243
>>>>>>>>            mc = (SSLModConfigRec *) 0x571738
>>>>>>>>            sc = (SSLSrvConfigRec *) 0x668c38
>>>>>>>>            pServ = (server_rec *) 0x65fa48
>>>>>>>>            cpVHostID = 0x60add0 "apache1.whatever.com:443"
>>>>>>>>            szPath =
>>>>>>>> "/apps/httpd2222/conf/certs/apache1.whatever.com.crt\000\177\000\000\000Ü׿\177\000\000\000à׿\177",
>>>>>>>> '\0'<repeats 15 times>,
>>>>>>>> "\001\000\000\000°å¿\177\000\000\000;\000\000\000+\000\000\000\020Û¿\177",
>>>>>>>> '\0'<repeats 35 times>, "à\224k", '\0'<repeats
13 times>,
>>>>>>>> "øé¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
>>>>>>>> '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°å¿\177",
>>>>>>>> '\0'<repeats 11 times>, "\020Û¿\177\000\000\000+\000\000\000;",
>>>>>>>> '\0'<repeats 35 times>...
>>>>>>>>            pPrivateKey = Variable "pPrivateKey" is not available.
>>>>>>>> (gdb)
>>>>>>>>
>>>>>>>>
>>>>>>>> Jim
>>>>>>>>
>>>>>>> I guess the thing that I'm puzzled about is why it goes from
>>>>>>> ssl_pphrase_Handle (which is part of Apache code?) to X509_free
()
>>>>>>> in the libobaccess.so?  Apache shouldn't even be aware of the
>>>>>>> libobaccess stuff yet (since I think that Apache is still trying
to
>>>>>>> initialize SSL processing)?
>>>>>>>
>>>>>>> Jim
>>>>>> It shouldn't unless you've declared a shutdown hook or a clean up
hook
>>>>>> of sorts. The looks of that stack trace look strikingly like an
>>>>>> exception handler firing off.  What are all of the hooks you have
>>>>>> running?
>>>>>>
>>>>>> Joe
>>>>>> --
>>>>>> http://www.silverhawk.net/
>>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'm basically using the original mod_headers.c as a starter code, and
>>>>> stuck some stuff in there.  Haven't done anything with the hooks etc.
>>>>> yet, so whatever was there is there:
>>>>>
>>>>> static void register_hooks(apr_pool_t *p)
>>>>> {
>>>>>       printf("mod_headers-jl V0.09 - start calling OAM API\n");
>>>>>       printf("In register_hooks\n");
>>>>>       ap_register_output_filter("FIXUP_HEADERS_OUT",
>>>>> ap_headers_output_filter,
>>>>>                                 NULL, AP_FTYPE_CONTENT_SET);
>>>>>       ap_register_output_filter("FIXUP_HEADERS_ERR",
>>>>> ap_headers_error_filter,
>>>>>                                 NULL, AP_FTYPE_CONTENT_SET);
>>>>>       ap_hook_pre_config(header_pre_config,NULL,NULL,APR_HOOK_MIDDLE);
>>>>>       ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
>>>>>       ap_hook_insert_filter(ap_headers_insert_output_filter, NULL,
>>>>> NULL, APR_HOOK_LAST);
>>>>>       ap_hook_insert_error_filter(ap_headers_insert_error_filter,
>>>>>                                   NULL, NULL, APR_HOOK_LAST);
>>>>>       ap_hook_fixups(ap_headers_fixup, NULL, NULL, APR_HOOK_LAST);
>>>>>       ap_hook_post_read_request(ap_headers_early, NULL, NULL,
>>>>> APR_HOOK_FIRST);
>>>>> }
>>>>>
>>>>> module AP_MODULE_DECLARE_DATA headers_module =
>>>>> {
>>>>>       STANDARD20_MODULE_STUFF,
>>>>>       create_headers_dir_config,  /* dir config creater */
>>>>>       merge_headers_config,       /* dir merger --- default is to
>>>>> override */
>>>>>       NULL,                       /* server config */
>>>>>       NULL,                       /* merge server configs */
>>>>>       headers_cmds,               /* command apr_table_t */
>>>>>       register_hooks              /* register hooks */
>>>>> };
>>>>>
>>>>> I guess that there is a:
>>>>>
>>>>> ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
>>>>>
>>>>> in that register_hooks(), but it was there from the original code.
>>>>>
>>>>> BTW, I was checking, and it looks like there's an X509_free() function
>>>>> in both openssl and in libobaccess.so (used 'nm --dynamic<file>),
so
>>>>> I'm wondering if things are confused, and maybe Apache code expects to
>>>>> call the openssl X509_free, but somehow ends up calling the
>>>>> X509_free() in libobaccess.so?
>>>>>
>>>>> What would cause something like that?  If two .so files have the same
>>>>> function, how to avoid conflicts?
>>>>>
>>>>> Jim
>>>>
>>>> Jim, you may have hit that on the head.  I'm not sure of the load order
>>>> and looking up symbols, but it could very well change things.  Try
>>>> re-ordering or adding a LoadFile before the SSL module and see if that
>>>> changes things.
>>>
>>> Look up order usually is load order. So if mod_ssl loads first and loads
>>> its dependency libcrypto, the symbol will always be found there even if
>>> the implementation in libobaccess.so is needed - and vice versa.
>>>
>>> On Solaris there is a -Bdirect linker flag which changes runtime search
>>> order so that each module would find a needed symbol first inits direct
>>> dependencies. Unfortunately AFAIK notihing similar exists for Linux.
>>>
>>> You would need to find ssl libs and libobaccess with compatible symbols.
>>> Best would be a libobaccess which is dynamically linked against a
>>> compatible version of libcrypto instead of - as it seems - being
>>> statically linked against it.
>>>
>>> Regards,
>>>
>>> Rainer
>>>
>>
>>
>> Hi Rainer,
>>
>> Here's exactly what I found:
>>
>> [root@apachemodule ~]# ldd /apps/netpoint/lib64/libobaccess.so
>>          libnsl.so.1 => /lib64/libnsl.so.1 (0x0000002a95ab8000)
>>          libdl.so.2 => /lib64/libdl.so.2 (0x0000002a95bd0000)
>>          libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x0000002a95cd3000)
>>          libstdc++.so.6 => /lib64/libstdc++.so.6 (0x0000002a95de9000)
>>          libm.so.6 => /lib64/tls/libm.so.6 (0x0000002a95fd9000)
>>          libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000002a9615f000)
>>          libc.so.6 => /lib64/tls/libc.so.6 (0x0000002a9626d000)
>>          /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
>> [root@apachemodule ~]#
>> [root@apachemodule ~]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "X509_free"
>> 00000000002d8d60 T X509_free
>>
>>
>> [root@apachemodule ~]# ldd /lib64/libssl.so.0.9.7a
>>          libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000003518700000)
>>          libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003519000000)
>>          libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003518500000)
>>          libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003518900000)
>>          libcrypto.so.4 => /lib64/libcrypto.so.4 (0x0000003518d00000)
>>          libdl.so.2 => /lib64/libdl.so.2 (0x0000003514800000)
>>          libz.so.1 => /usr/lib64/libz.so.1 (0x0000003514c00000)
>>          libc.so.6 => /lib64/tls/libc.so.6 (0x0000003514300000)
>>          libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003518300000)
>>          /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
>> [root@apachemodule ~]#
>> [root@apachemodule ~]# nm --dynamic /lib64/libssl.so.0.9.7a | grep "X509_free"
>>                   U X509_free
>>
>>
>> [root@apachemodule ~]# ldd /lib64/libcrypto.so.0.9.7a
>>          libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000003518700000)
>>          libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003519000000)
>>          libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003518500000)
>>          libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003518900000)
>>          libdl.so.2 => /lib64/libdl.so.2 (0x0000003514800000)
>>          libz.so.1 => /usr/lib64/libz.so.1 (0x0000003514c00000)
>>          libc.so.6 => /lib64/tls/libc.so.6 (0x0000003514300000)
>>          libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003518300000)
>>          /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
>> [root@apachemodule ~]#
>> [root@apachemodule ~]# nm --dynamic /lib64/libcrypto.so.0.9.7a | grep "X509_free"
>> 0000003518da9860 T X509_free
>>
>>
>> So:
>>
>> - liboaccess.so doesn't REFERENCE libcrypto, but it HAS an X509_free() function in
it.
>> - libssl.so.REFERENCES libcrypto, AND it HAS an X509_free function in it.
>> - libcrypto also has an X509_free function in it.
>>
>>  From your msg above, are you saying that this situation can't be resolved by trying
to change load order?
>>
>> And the only way is what you said in your last paragraph (find another libobccess.so)?

I don't know a way how to fix this apart from using mod_ssl and 
libobccess which have both bin build against the same major version of 
OpenSSL. In your case mod_ssl seems to use the platform OpenSSL, so 
0.9.7(a) (which is pretty old), and we don't know what OpenSSL version 
was linked in statically into your libobccess.so.

Load order most likely doesn't help. Either mod_header/liboccess.so will 
habe a problem or mod_ssl.

> Also, BTW, the call to X509_free is apparently because in ssl_engine_pphrase.c, which
it seems is part of mod_ssl, it has:
>
>              /*
>               * Free the X509 structure
>               */
>              X509_free(pX509Cert);
>
> but that's calling the X509_free() in libobaccess.so, rather than the X509_free() in
libcrypt.so?

It's just calling X509_free(). The runtime linker will search through 
the httpd binary, all loaded modules and libraries in load order and 
call the symbol wherever it found it first. So you can change load order 
and load mod_ssl and OpenSSL libs before libobccess.so. But then when 
libobccess runs it will itself call symbols from within OpenSSL, e.g. 
X509_free() and again the runtime linker will search for the symbol 
through the loaded files *and not dircetly use the one inside libobccess 
- even if it is caled form inside this file*.

Regards,

Rainer

Mime
View raw message