httpd-bugs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject [Bug 60071] Child httpd processes crash with Segmentation fault when enabling more than 1 healthcheck
Date Wed, 18 Jan 2017 10:18:21 GMT
https://bz.apache.org/bugzilla/show_bug.cgi?id=60071

--- Comment #26 from Endika <egandarias@gfi.es> ---
Hello,
We have tested the patch today and in less than a hour we've obtained 8
Segmentation Faults...

Logs in error log:

[Wed Jan 18 11:02:01.358056 2017] [proxy:debug] [pid 32592:tid 140559909840640]
proxy_util.c(2209): [client 10.136.101.140:62325] AH00944: connecting
http://ejld1131.ejgvdns:8170/webhdfs/v1/data?op=GETFILESTATUS&user.name=root to
ejld1131.ejgvdns:8170
[Wed Jan 18 11:02:01.358067 2017] [proxy:debug] [pid 32592:tid 140559909840640]
proxy_util.c(2418): [client 10.136.101.140:62325] AH00947: connected
/webhdfs/v1/data?op=GETFILESTATUS&user.name=root to ejld1131.ejgvdns:8170
[Wed Jan 18 11:02:01.359566 2017] [proxy:debug] [pid 32592:tid 140559909840640]
proxy_util.c(2171): AH00943: http: has released connection for
(ejld1131.ejgvdns)
[Wed Jan 18 11:02:01.359590 2017] [proxy_balancer:debug] [pid 32592:tid
140559909840640] mod_proxy_balancer.c(688): [client 10.136.101.140:62325]
AH01176: proxy_balancer_post_request for (balancer://cluster_namenode)
[Wed Jan 18 11:02:01.373675 2017] [core:notice] [pid 29786:tid 140560333223744]
AH00051: child pid 4685 exit signal Segmentation fault (11), possible coredump
in /tmp
[Wed Jan 18 11:02:10.776046 2017] [proxy_balancer:debug] [pid 9243:tid
140560027338496] mod_proxy_balancer.c(688): [client 10.136.101
.140:62342] AH01176: proxy_balancer_post_request for
(balancer://cluster_namenode)
[Wed Jan 18 11:02:10.794876 2017] [watchdog:debug] [pid 3222:tid
140560165148416] mod_watchdog.c(159): AH02972: Singleton Watchdog
(_proxy_hcheck_) running
[Wed Jan 18 11:02:10.794991 2017] [proxy_hcheck:debug] [pid 3222:tid
140560165148416] mod_proxy_hcheck.c(888): AH03258: _proxy_hcheck_ watchdog
started.
[Wed Jan 18 11:02:10.796926 2017] [proxy_hcheck:debug] [pid 3222:tid
140560165148416] mod_proxy_hcheck.c(902): AH03313: apr_thread_pool_create()
with 16 threads succeeded
[Wed Jan 18 11:02:10.979604 2017] [authz_core:debug] [pid 9243:tid
140560027338496] mod_authz_core.c(835): [client 10.136.101.140:62342] AH01628:
authorization result: granted (no directives)
[Wed Jan 18 11:02:12.800263 2017] [proxy:debug] [pid 3222:tid 140560165148416]
proxy_util.c(1856): AH00930: initialized pool in child
 3222 for (ejld1132.ejgvdns) min=0 max=25 smax=25
[Wed Jan 18 11:02:12.800440 2017] [proxy_hcheck:debug] [pid 3222:tid
140560165148416] mod_proxy_hcheck.c(457): AH03248: Creating hc worker 20a0690
for http://ejld1132.ejgvdns:8170
[Wed Jan 18 11:02:12.800479 2017] [proxy:debug] [pid 3222:tid 140560165148416]
proxy_util.c(1779): AH00925: initializing worker 20a0690 shared
[Wed Jan 18 11:02:12.800488 2017] [proxy:debug] [pid 3222:tid 140560165148416]
proxy_util.c(1821): AH00927: initializing worker 20a0690 local
[Wed Jan 18 11:02:12.800513 2017] [proxy:debug] [pid 3222:tid 140560165148416]
proxy_util.c(1856): AH00930: initialized pool in child 3222 for
(ejld1132.ejgvdns) min=0 max=25 smax=25
[Wed Jan 18 11:02:12.800571 2017] [proxy_hcheck:debug] [pid 3222:tid
140560165148416] mod_proxy_hcheck.c(457): AH03248: Creating hc worker 20a0c70
for http://ejld1131.ejgvdns:8170
[Wed Jan 18 11:02:12.800595 2017] [proxy:debug] [pid 3222:tid 140560165148416]
proxy_util.c(1779): AH00925: initializing worker 20a0c70 shared
[Wed Jan 18 11:02:12.800603 2017] [proxy:debug] [pid 3222:tid 140560165148416]
proxy_util.c(1821): AH00927: initializing worker 20a0c70 local
[Wed Jan 18 11:02:12.800619 2017] [proxy:debug] [pid 3222:tid 140560165148416]
proxy_util.c(1856): AH00930: initialized pool in child 3222 for
(ejld1131.ejgvdns) min=0 max=25 smax=25
[Wed Jan 18 11:02:12.800623 2017] [proxy_hcheck:debug] [pid 3222:tid
140559523972864] mod_proxy_hcheck.c(819): AH03256: Threaded Health checking
http://ejld1132.ejgvdns:8170
[Wed Jan 18 11:02:12.800710 2017] [proxy_hcheck:debug] [pid 3222:tid
140559641470720] mod_proxy_hcheck.c(819): AH03256: Threaded Health checking
http://ejld1131.ejgvdns:8170
[Wed Jan 18 11:02:12.800874 2017] [proxy:debug] [pid 3222:tid 140559523972864]
proxy_util.c(2156): AH00942: HCOH: has acquired connection for
(ejld1132.ejgvdns)
[Wed Jan 18 11:02:12.800900 2017] [proxy:debug] [pid 3222:tid 140559641470720]
proxy_util.c(2156): AH00942: HCOH: has acquired connection for
(ejld1131.ejgvdns)
[Wed Jan 18 11:02:12.801249 2017] [proxy:debug] [pid 3222:tid 140559523972864]
proxy_util.c(2884): AH02824: HCOH: connection established with
10.190.40.107:8170 (ejld1132.ejgvdns)
[Wed Jan 18 11:02:12.801293 2017] [proxy:debug] [pid 3222:tid 140559523972864]
proxy_util.c(3051): AH00962: HCOH: connection complete to 10.190.40.107:8170
(ejld1132.ejgvdns)
[Wed Jan 18 11:02:12.801412 2017] [proxy:debug] [pid 3222:tid 140559641470720]
proxy_util.c(2884): AH02824: HCOH: connection established with
10.190.40.106:8170 (ejld1131.ejgvdns)
[Wed Jan 18 11:02:12.801445 2017] [proxy:debug] [pid 3222:tid 140559641470720]
proxy_util.c(3051): AH00962: HCOH: connection complete to 10.190.40.106:8170
(ejld1131.ejgvdns)
[Wed Jan 18 11:02:12.805093 2017] [proxy_hcheck:debug] [pid 3222:tid
140559523972864] mod_proxy_hcheck.c(626): AH03254: HTTP/1.1 403 Forbidden
[Wed Jan 18 11:02:12.805173 2017] [proxy:debug] [pid 3222:tid 140559523972864]
proxy_util.c(2171): AH00943: HCOH: has released connection for
(ejld1132.ejgvdns)
[Wed Jan 18 11:02:12.805278 2017] [proxy_hcheck:debug] [pid 3222:tid
140559523972864] mod_proxy_hcheck.c(560): AH03251: Health check GET Status (1)
for 20a0690.
[Wed Jan 18 11:02:12.806221 2017] [proxy_hcheck:debug] [pid 3222:tid
140559641470720] mod_proxy_hcheck.c(626): AH03254: HTTP/1.1 200 OK
[Wed Jan 18 11:02:12.806274 2017] [proxy:debug] [pid 3222:tid 140559641470720]
proxy_util.c(2171): AH00943: HCOH: has released connection for
(ejld1131.ejgvdns)
[Wed Jan 18 11:02:12.806331 2017] [proxy_hcheck:debug] [pid 3222:tid
140559641470720] mod_proxy_hcheck.c(560): AH03251: Health check GET Status (0)
for 20a0c70.
[Wed Jan 18 11:02:12.822328 2017] [authz_core:debug] [pid 9243:tid
140560027338496] mod_authz_core.c(835): [client 10.136.101.140:62342] AH01628:
authorization result: granted (no directives)
[Wed Jan 18 11:02:12.822448 2017] [lbmethod_byrequests:debug] [pid 9243:tid
140560027338496] mod_lbmethod_byrequests.c(97): AH01207: proxy: Entering
byrequests for BALANCER (balancer://cluster_namenode)
[Wed Jan 18 11:02:12.822458 2017] [lbmethod_byrequests:debug] [pid 9243:tid
140560027338496] mod_lbmethod_byrequests.c(144): AH01208: proxy: byrequests
selected worker "http://ejld1131.ejgvdns:8170" : busy 0 : lbstatus 0
[Wed Jan 18 11:02:12.822517 2017] [proxy_balancer:debug] [pid 9243:tid
140560027338496] mod_proxy_balancer.c(632): [client 10.136.101.140:62342]
AH01172: balancer://cluster_namenode: worker (http://ejld1131.ejgvdns:8170)
rewritten to
http://ejld1131.ejgvdns:8170/webhdfs/v1/data?op=GETFILESTATUS&user.name=root



Core dump

(gdb) bt full
#0  ap_core_output_filter (f=0x7fd6800090f8, new_bb=0x7fd688000a60) at
core_filters.c:491
        c = 0x7fd680008bf0
        net = 0x7fd6800090b0
        ctx = 0x7fd680009140
        bb = 0x7fd688000a60
        bucket = 0x1
        next = <optimized out>
        flush_upto = 0x1
        bytes_in_brigade = 0
        non_file_bytes_in_brigade = 0
        eor_buckets_in_brigade = 0
        morphing_bucket_in_brigade = 0
        rv = <optimized out>
#1  0x00007fd6b7976c03 in hc_send (ctx=0x2085678, ctx=0x2085678,
backend=0x7fd69c026e80, out=<optimized out>, ptemp=0x7fd6880008e8)
    at mod_proxy_hcheck.c:612
        tmp_bb = 0x7fd688000a60
#2  hc_check_http (baton=0x7fd688000960) at mod_proxy_hcheck.c:771
        backend = 0x7fd69c026e80
        ctx = 0x2085678
        status = <optimized out>
        ptemp = 0x7fd6880008e8
        method = 0x7fd6b7978438 "GET"
        hc = 0x2088528
        worker = 0x20a0690
        c = {pool = 0x0, base_server = 0x0, vhost_lookup_data = 0x0, local_addr
= 0x0, client_addr = 0x0, client_ip = 0x0,
          remote_host = 0x0, remote_logname = 0x0, local_ip = 0x7fd69c020cb0
"\370\f\002\234\326\177",
          local_host = 0x3f1713a <Address 0x3f1713a out of bounds>, id =
140559784145016, conn_config = 0x7fd69c020ce0,
          notes = 0x7fd69c020c10, input_filters = 0x7fd6c0119876
<find_entry+134>, output_filters = 0x7fd6a000f0d8, sbh = 0x14,
          bucket_alloc = 0x0, cs = 0x7fd69c020c10, data_in_input_filters =
-1610551080, data_in_output_filters = 32726,
          clogging_input_filters = 0, double_reverse = -1, aborted = 32726,
keepalive = (unknown: 2684416216), keepalives = 32726,
          log = 0x7fd6907f0700, log_id = 0x1 <Address 0x1 out of bounds>,
current_thread = 0x7fd6c0119c2f <apr_hash_set+15>,
          master = 0x7fd69c020b98}
        wctx = <optimized out>
        cond = <optimized out>
#3  hc_check (thread=0x7fd6a000edc8, b=0x7fd688000960) at
mod_proxy_hcheck.c:829
---Type <return> to continue, or q <return> to quit---
        baton = 0x7fd688000960
        s = 0x203b5e8
        worker = 0x20a0690
        now = 1484733718004809
        rv = <optimized out>
#4  0x00007fd6c057aedd in thread_pool_func (t=0x7fd6a000edc8, param=0x2088390)
at misc/apr_thread_pool.c:266
        me = 0x2088390
        task = 0x7fd6a000f0d8
        wait = <optimized out>
        __PRETTY_FUNCTION__ = "thread_pool_func"
#5  0x00007fd6bfab3dc5 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#6  0x00007fd6bf5dd28d in clone () from /lib64/libc.so.6

Thanks

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


Mime
View raw message