httpd-modules-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <oh...@cox.net>
Subject Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use Apache module that uses shared library?
Date Thu, 01 Jan 1970 00:00:00 GMT

---- Joe Lewis <joe@joe-lewis.com> wrote: 
> On 6/21/12 9:39 PM, ohaya@cox.net wrote:
> > ---- ohaya@cox.net wrote:
> >> ---- ohaya@cox.net wrote:
> >>> ---- Joe Lewis<joe@joe-lewis.com>  wrote:
> >>>> On 6/21/12 7:32 PM, ohaya@cox.net wrote:
> >>>>> ---- ohaya@cox.net wrote:
> >>>>>> ---- Joe Lewis<joe@joe-lewis.com>   wrote:
> >>>>>>> On 6/21/12 6:46 PM, ohaya@cox.net wrote:
> >>>>>>>> ---- Joe Lewis<joe@joe-lewis.com>    wrote:
> >>>>>>>>> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
> >>>>>>>>>> ---- ohaya@cox.net wrote:
> >>>>>>>>>>> ---- Sorin Manolache<sorinm@gmail.com>
    wrote:
> >>>>>>>>>>>> And I forgot to say: run gdb in some
sort of environment where you see
> >>>>>>>>>>>> your current source code line and a
couple of surrounding lines. You
> >>>>>>>>>>>> could achieve this with the "list" command,
but I prefer running gdb in
> >>>>>>>>>>>> emacs and let emacs do the nice listing
of source code in a different panel.
> >>>>>>>>>>>>
> >>>>>>>>>>>> S
> >>>>>>>>>>> Here's the function from my source.  It's
the original from mod_headers.c, plus my printf:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> static int header_post_config(apr_pool_t
*pconf, apr_pool_t *plog,
> >>>>>>>>>>>                                   apr_pool_t
*ptemp, server_rec *s)
> >>>>>>>>>>> {
> >>>>>>>>>>>         printf("In header_post_config\n");
> >>>>>>>>>>>         header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>>>>>         return OK;
> >>>>>>>>>>> }
> >>>>>>>>>>>
> >>>>>>>>>>> Jim
> >>>>>>>>>> Hi,
> >>>>>>>>>>
> >>>>>>>>>> I was able to get the segfault to go away. 
Here's what I had to do:
> >>>>>>>>>>
> >>>>>>>>>> - Created /etc/ld.so.conf.d/my.conf, and added
the directory where my libobaccess.so was
> >>>>>>>>>> - Run 'ldconfig' to activate.
> >>>>>>>>>> - In the apxs command, DON'T include the -L
and -l arguments
> >>>>>>>>>>
> >>>>>>>>>> After that, Apache appears to start ok, without
segfault :)!!
> >>>>>>>>>>
> >>>>>>>>>> Thanks for all of the great help, esp. the suggestion
about checking "ldconfig -p".  I still don't understand why, but I'm just glad that I can
get past this piece so now I can debug my module :)...
> >>>>>>>>>>
> >>>>>>>>>> Later,
> >>>>>>>>>> Jim
> >>>>>>>>> I'm just glad this list is as good as it is!
> >>>>>>>>>
> >>>>>>>>> FYI, the ldconfig is the dynamic linker control,
and those
> >>>>>>>>> /etc/ld.so.conf.d files provide additional search
directories for the
> >>>>>>>>> linker to check in when loading a library.
> >>>>>>>>>
> >>>>>>>>> Joe
> >>>>>>>>> --
> >>>>>>>>> http://www.silverhawk.net/
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> Sorry to report, but my earlier report was a "false
positive" :)...
> >>>>>>>>
> >>>>>>>> I forgot that the mod_headers.c that I was doing the
earlier testing with had all references to the libobaccess.so removed :(!!
> >>>>>>>>
> >>>>>>>> So, I'm still stuck with basically the same problem,
now, working with my "full" code, with the calls in it:
> >>>>>>>>
> >>>>>>>> - If I compile with -L and -l, Apache segfaults when
it starts
> >>>>>>>>
> >>>>>>>> - If I compile without -L and -l, then I get "undefined
symbol" errors when I try to start Apache, e.g.:
> >>>>>>>>
> >>>>>>>> [root@apachemodule build-mod_headers]# /apps/httpd2222/bin/apachectl
-k start -X
> >>>>>>>> httpd: Syntax error on line 84 of /apps/httpd2222/conf/httpd.conf:
Cannot load /apps/httpd2222/modules/mod_headers.so into server: /apps/httpd2222/modules/mod_headers.so:
undefined symbol: ObResource_isProtected
> >>>>>>>>
> >>>>>>>> That "ObResource_isProtected" should be a symbol in
libobaccess.so, and in fact, if I do "nm --dynamic", I get:
> >>>>>>>>
> >>>>>>>> [root@apachemodule build-mod_headers]# nm --dynamic
/apps/netpoint/lib64/libobaccess.so | grep "ObResource_isProtected"
> >>>>>>>> 00000000000a6d80 T ObResource_isProtected
> >>>>>>>> [root@apachemodule build-mod_headers]#
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> I'm *assuming* that the reason for the "undefined symbol"
error is that libobaccess.so is actually not being loaded, but then when I try to load libobaccess.so,
either via -L and -l in the apxs, or using LoadFile in httpd.conf, I get the segfault (same
gdb info, BTW).
> >>>>>>>>
> >>>>>>>> Catch-22?
> >>>>>>>>
> >>>>>>>> Sorry for the false alarm :(!!
> >>>>>>>>
> >>>>>>>> Jim
> >>>>>>>>
> >>>>>>>>
> >>>>>>> Not a catch-22.  The -L and -l specify linker options when
assembling
> >>>>>>> the code.  The ldconfig is a run-time thing.  If you are
getting the
> >>>>>>> stderr messages, you are making it all the way into your
library.  I'd
> >>>>>>> suggest commenting out the following line and see if you
get farther :
> >>>>>>>
> >>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>
> >>>>>>> That should tell you if the problem is the ssl_var_lookup.
> >>>>>>>
> >>>>>>> Joe
> >>>>>>> --
> >>>>>>> http://www.silverhawk.net/
> >>>>>> Hi,
> >>>>>>
> >>>>>> Thanks for the suggestion.  I just tried what you suggested,
and got a segfault when I started Apache with the modified module.
> >>>>>>
> >>>>>> Jim
> >>>>>>
> >>>>>>
> >>>>> Hi,
> >>>>>
> >>>>> As a reminder, here's the gdb with the library loaded:
> >>>>>
> >>>>>
> >>>>> (gdb) b header_post_config
> >>>>> Function "header_post_config" not defined.
> >>>>> Make breakpoint pending on future shared library load? (y or [n])
y
> >>>>> Breakpoint 1 (header_post_config) pending.
> >>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> >>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/
-f /apps/httpd2222/conf/httpd.conf
> >>>>> [Thread debugging using libthread_db enabled]
> >>>>> [New Thread 182897610272 (LWP 11317)]
> >>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> >>>>> Pending breakpoint "header_post_config" resolved
> >>>>> mod_headers-jl V0.09 - start calling OAM API
> >>>>> In register_hooks
> >>>>> In create_headers_dir_config
> >>>>> In create_headers_dir_config
> >>>>> In header_cmd
> >>>>> In header_inout_cmd
> >>>>> In parse_format_tag
> >>>>> In parse_misc_string
> >>>>> In create_headers_dir_config
> >>>>> In header_cmd
> >>>>> In header_inout_cmd
> >>>>> In parse_format_tag
> >>>>> In parse_misc_string
> >>>>> [Switching to Thread 182897610272 (LWP 11317)]
> >>>>>
> >>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> >>>>> 1121        printf("In header_post_config\n");
> >>>>> (gdb) n
> >>>>> 1120    {
> >>>>> (gdb) n
> >>>>> 1121        printf("In header_post_config\n");
> >>>>> (gdb) n
> >>>>> In header_post_config
> >>>>> 1122        header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>> (gdb) n
> >>>>> 1124    }
> >>>>> (gdb) n
> >>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8,
ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> >>>>> 91      AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> >>>>> (gdb) n
> >>>>>
> >>>>> Program received signal SIGSEGV, Segmentation fault.
> >>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>>
> >>>>>
> >>>>> So, it's actually blowing up in "BN_num_bits()" in /lib64/libcrypto.so.4?
> >>>>>
> >>>>> Jim
> >>>> When you see the :
> >>>>
> >>>> Program received signal SIGSEGV, Segmentation fault.
> >>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>
> >>>>
> >>>> in gdb, type "bt" and hit enter.  It should show the back trace and
how
> >>>> you got to the BN_num_bits() function.
> >>>>
> >>>> Joe
> >>>> --
> >>>> http://www.silverhawk.net/
> >>>
> >>> Hi,
> >>>
> >>> Here's the bt full:
> >>>
> >>>
> >>> In header_inout_cmd
> >>> In parse_format_tag
> >>> In parse_misc_string
> >>> [Switching to Thread 182897610272 (LWP 6676)]
> >>>
> >>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8,
s=0x59d3a8) at mod_headers.c:1121
> >>> 1121        printf("In header_post_config\n");
> >>> (gdb) n
> >>> 1120    {
> >>> (gdb) n
> >>> 1121        printf("In header_post_config\n");
> >>> (gdb) n
> >>> In header_post_config
> >>> 1122        header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>> (gdb) n
> >>> 1124    }
> >>> (gdb) n
> >>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8,
ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> >>> 91      AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> >>> (gdb) n
> >>>
> >>> Program received signal SIGSEGV, Segmentation fault.
> >>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>> (gdb) bt full
> >>> #0  0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>> No symbol table info available.
> >>> #1  0x0000003518da8f4e in X509_ATTRIBUTE_create () from /lib64/libcrypto.so.4
> >>> No symbol table info available.
> >>> #2  0x0000003518dadea2 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> >>> No symbol table info available.
> >>> #3  0x0000003518dadf79 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> >>> No symbol table info available.
> >>> #4  0x0000003518dae0e1 in ASN1_item_ex_i2d () from /lib64/libcrypto.so.4
> >>> No symbol table info available.
> >>> #5  0x0000003518dae5f2 in ASN1_template_i2d () from /lib64/libcrypto.so.4
> >>> No symbol table info available.
> >>> #6  0x0000003518dae28e in ASN1_item_ex_i2d () from /lib64/libcrypto.so.4
> >>> No symbol table info available.
> >>> #7  0x0000003518dae3c6 in ASN1_item_i2d () from /lib64/libcrypto.so.4
> >>> No symbol table info available.
> >>> #8  0x0000002a987d9d3a in ssl_pphrase_Handle (s=0x59d3a8, p=0x5a72d8) at
ssl_engine_pphrase.c:505
> >>>          mc = (SSLModConfigRec *) 0x571738
> >>>          sc = (SSLSrvConfigRec *) 0x668c38
> >>>          pServ = (server_rec *) 0x65fa48
> >>>          cpVHostID = 0x60add0 "www.example.com:443"
> >>>          szPath = "/apps/httpd2222/conf/certs/apache1.whatever.com.key\000\177\000\000\000Ü×ÿ¿\177\000\000\000à×ÿ¿\177",
'\0'<rep                                                eats 15 times>, "\001\000\000\000°åÿ¿\177\000\000\000;\000\000\000+\000\000\000\020Ûÿ¿\177",
'\0'<repeats 35 times>, "à\224k", '                                               
---Type<return>  to continue, or q<return>  to quit---
> >>> \0'<repeats 13 times>, "øéÿ¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
'\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°åÿ¿\177", '\0'<repeats
11 times>, "\020Ûÿ¿\177\000\000\000+\000\000\000;", '\0'<repeats 35 times>...
> >>>          pPrivateKey = (EVP_PKEY *) 0x6ba670
> >>>          asn1 = Variable "asn1" is not available.
> >>> (gdb)
> >>>
> >>>
> >>> Not really sure what to make of that though :(...
> >>>
> >>> Jim
> >>
> >> Hi,
> >>
> >> Ok, something interesting.  I noticed that the servername in the gdb output
was example.whatever.com, so I changed ServerName in extras/httpd-ssl.conf to "apache1.whatever.com".
 Apache still segfaulted after that.
> >>
> >> I noticed that igdb was showing my router IP address for "mod_unique_id" (whatever
that is), so I thought it was because I didn't have the hostname in /etc/hosts.  So, I added
"apache1.whatever.com" to /etc/hosts, and also turned Apache LogLevel to debug, and now I
get a different error:
> >>
> >> [root@apachemodule dev]# /apps/httpd2222/bin/apachectl -k start -X
> >> mod_headers-jl V0.09 - start calling OAM API
> >> In register_hooks
> >> In create_headers_dir_config
> >> In create_headers_dir_config
> >> In header_cmd
> >> In header_inout_cmd
> >> In parse_format_tag
> >> In parse_misc_string
> >> In create_headers_dir_config
> >> In header_cmd
> >> In header_inout_cmd
> >> In parse_format_tag
> >> In parse_misc_string
> >> In header_post_config
> >> *** glibc detected *** corrupted double-linked list: 0x00000000006b9710 ***
> >> /apps/httpd2222/bin/apachectl: line 78:  7599 Aborted                 $HTTPD
$ARGV
> >> [root@apachemodule dev]#
> >>
> >>
> >> Here's the Apache error_log:
> >>
> >> [Thu Jun 21 23:24:11 2012] [info] mod_unique_id: using ip addr 72.215.225.9
> >> [Thu Jun 21 23:24:12 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
> >> [Thu Jun 21 23:24:12 2012] [info] Loading certificate&  private key of SSL-aware
server
> >> [Thu Jun 21 23:29:51 2012] [info] mod_unique_id: using ip addr 72.215.225.9
> >> [Thu Jun 21 23:29:52 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
> >> [Thu Jun 21 23:29:52 2012] [info] Loading certificate&  private key of SSL-aware
server
> >> [Thu Jun 21 23:29:56 2012] [info] mod_unique_id: using ip addr 72.215.225.9
> >> [Thu Jun 21 23:29:57 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
> >> [Thu Jun 21 23:29:57 2012] [info] Loading certificate&  private key of SSL-aware
server
> >> [root@apachemodule dev]#
> >>
> >>
> >> and here's the gdb backtrace:
> >>
> >> (gdb) b header_post_config
> >> Function "header_post_config" not defined.
> >> Make breakpoint pending on future shared library load? (y or [n]) y
> >> Breakpoint 1 (header_post_config) pending.
> >> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> >> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> >> [Thread debugging using libthread_db enabled]
> >> [New Thread 182897610272 (LWP 7644)]
> >> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> >> Pending breakpoint "header_post_config" resolved
> >> mod_headers-jl V0.09 - start calling OAM API
> >> In register_hooks
> >> In create_headers_dir_config
> >> In create_headers_dir_config
> >> In header_cmd
> >> In header_inout_cmd
> >> In parse_format_tag
> >> In parse_misc_string
> >> In create_headers_dir_config
> >> In header_cmd
> >> In header_inout_cmd
> >> In parse_format_tag
> >> In parse_misc_string
> >> [Switching to Thread 182897610272 (LWP 7644)]
> >>
> >> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8,
s=0x59d3a8) at mod_headers.c:1121
> >> 1121        printf("In header_post_config\n");
> >> (gdb) n
> >> 1120    {
> >> (gdb) n
> >> 1121        printf("In header_post_config\n");
> >> (gdb) n
> >> In header_post_config
> >> 1122        header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >> (gdb) n
> >> 1124    }
> >> (gdb) n
> >> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8,
s=0x59d3a8) at config.c:91
> >> 91      AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> >> (gdb) n
> >> *** glibc detected *** corrupted double-linked list: 0x00000000006b9710 ***
> >>
> >> Program received signal SIGABRT, Aborted.
> >> 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
> >> (gdb) bt full
> >> #0  0x000000351432e26d in raise () from /lib64/tls/libc.so.6
> >> No symbol table info available.
> >> #1  0x000000351432fa6e in abort () from /lib64/tls/libc.so.6
> >> No symbol table info available.
> >> #2  0x0000003514363641 in __libc_message () from /lib64/tls/libc.so.6
> >> No symbol table info available.
> >> #3  0x0000003514369512 in _int_free () from /lib64/tls/libc.so.6
> >> No symbol table info available.
> >> #4  0x0000003514369846 in free () from /lib64/tls/libc.so.6
> >> No symbol table info available.
> >> #5  0x0000002a9790b6ba in R_free () from /apps/netpoint/lib64/libobaccess.so
> >> No symbol table info available.
> >> #6  0x0000002a9792bc41 in X509_CINF_free () from /apps/netpoint/lib64/libobaccess.so
> >> No symbol table info available.
> >> #7  0x0000002a9790ed98 in X509_free () from /apps/netpoint/lib64/libobaccess.so
> >> No symbol table info available.
> >> #8  0x0000002a987d97b6 in ssl_pphrase_Handle (s=0x59d3a8, p=0x5a72d8) at ssl_engine_pphrase.c:243
> >>          mc = (SSLModConfigRec *) 0x571738
> >>          sc = (SSLSrvConfigRec *) 0x668c38
> >>          pServ = (server_rec *) 0x65fa48
> >>          cpVHostID = 0x60add0 "apache1.whatever.com:443"
> >>          szPath = "/apps/httpd2222/conf/certs/apache1.whatever.com.crt\000\177\000\000\000Ü×ÿ¿\177\000\000\000à×ÿ¿\177",
'\0'<repeats 15 times>, "\001\000\000\000°åÿ¿\177\000\000\000;\000\000\000+\000\000\000\020Ûÿ¿\177",
'\0'<repeats 35 times>, "à\224k", '\0'<repeats 13 times>, "øéÿ¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
'\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°åÿ¿\177", '\0'<repeats
11 times>, "\020Ûÿ¿\177\000\000\000+\000\000\000;", '\0'<repeats 35 times>...
> >>          pPrivateKey = Variable "pPrivateKey" is not available.
> >> (gdb)
> >>
> >>
> >> Jim
> >>
> >
> > I guess the thing that I'm puzzled about is why it goes from  ssl_pphrase_Handle
(which is part of Apache code?) to X509_free () in the libobaccess.so?  Apache shouldn't even
be aware of the libobaccess stuff yet (since I think that Apache is still trying to initialize
SSL processing)?
> >
> > Jim
> 
> It shouldn't unless you've declared a shutdown hook or a clean up hook 
> of sorts. The looks of that stack trace look strikingly like an 
> exception handler firing off.  What are all of the hooks you have running?
> 
> Joe
> --
> http://www.silverhawk.net/
> 


Hi,

I'm basically using the original mod_headers.c as a starter code, and stuck some stuff in
there.  Haven't done anything with the hooks etc. yet, so whatever was there is there:

static void register_hooks(apr_pool_t *p)
{
    printf("mod_headers-jl V0.09 - start calling OAM API\n");
    printf("In register_hooks\n");
    ap_register_output_filter("FIXUP_HEADERS_OUT", ap_headers_output_filter,
                              NULL, AP_FTYPE_CONTENT_SET);
    ap_register_output_filter("FIXUP_HEADERS_ERR", ap_headers_error_filter,
                              NULL, AP_FTYPE_CONTENT_SET);
    ap_hook_pre_config(header_pre_config,NULL,NULL,APR_HOOK_MIDDLE);
    ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
    ap_hook_insert_filter(ap_headers_insert_output_filter, NULL, NULL, APR_HOOK_LAST);
    ap_hook_insert_error_filter(ap_headers_insert_error_filter,
                                NULL, NULL, APR_HOOK_LAST);
    ap_hook_fixups(ap_headers_fixup, NULL, NULL, APR_HOOK_LAST);
    ap_hook_post_read_request(ap_headers_early, NULL, NULL, APR_HOOK_FIRST);
}

module AP_MODULE_DECLARE_DATA headers_module =
{
    STANDARD20_MODULE_STUFF,
    create_headers_dir_config,  /* dir config creater */
    merge_headers_config,       /* dir merger --- default is to override */
    NULL,                       /* server config */
    NULL,                       /* merge server configs */
    headers_cmds,               /* command apr_table_t */
    register_hooks              /* register hooks */
};

I guess that there is a:

ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);

in that register_hooks(), but it was there from the original code.

BTW, I was checking, and it looks like there's an X509_free() function in both openssl and
in libobaccess.so (used 'nm --dynamic <file>), so I'm wondering if things are confused,
and maybe Apache code expects to call the openssl X509_free, but somehow ends up calling the
X509_free() in libobaccess.so?

What would cause something like that?  If two .so files have the same function, how to avoid
conflicts?

Jim

Mime
View raw message