Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 7E13B200C29 for ; Tue, 28 Feb 2017 11:50:07 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 7AEFE160B7C; Tue, 28 Feb 2017 10:50:07 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 08935160B6A for ; Tue, 28 Feb 2017 11:50:05 +0100 (CET) Received: (qmail 85544 invoked by uid 500); 28 Feb 2017 10:50:05 -0000 Mailing-List: contact dev-help@httpd.apache.org; run by ezmlm Precedence: bulk Reply-To: dev@httpd.apache.org list-help: list-unsubscribe: List-Post: List-Id: Delivered-To: mailing list dev@httpd.apache.org Received: (qmail 85534 invoked by uid 99); 28 Feb 2017 10:50:05 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Feb 2017 10:50:05 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id A175BC094B for ; Tue, 28 Feb 2017 10:50:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -3.099 X-Spam-Level: X-Spam-Status: No, score=-3.099 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RP_MATCHES_RCVD=-2.999, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=greenbytes.de header.b=qXrPaaeV; dkim=pass (1024-bit key) header.d=greenbytes.de header.b=qXrPaaeV Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 2MJdEd6VYQWP for ; Tue, 28 Feb 2017 10:50:01 +0000 (UTC) Received: from mail.greenbytes.de (mail.greenbytes.de [5.10.171.186]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 616DE5F23A for ; Tue, 28 Feb 2017 10:50:00 +0000 (UTC) Received: by mail.greenbytes.de (Postfix, from userid 117) id EA7CA15A0874; Tue, 28 Feb 2017 11:49:52 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=greenbytes.de; s=mail; t=1488278992; bh=0zohcgo8gZ0/vDSvu1H6x1VPlQjggPE5X6zBapo7Au8=; h=From:Subject:Date:References:To:In-Reply-To:From; b=qXrPaaeVdYI+naXGL6drJkBWyO2JRfGmIJ1VgSBGVfeUYiXoaifhyXpuuHg7ANFCr TtEZXCYQNsNqBToaUwUtEUZxeaIdpnhyHdVceDJApQkhZJL5Kc3lKS7z9YukrozGNv XLe5q54i95pIdZG5FesNqQV768XKz6FmNTtrPmLY= Received: from [192.168.178.72] (unknown [84.150.94.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mail.greenbytes.de (Postfix) with ESMTPSA id EF35B15A0252 for ; Tue, 28 Feb 2017 11:49:51 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=greenbytes.de; s=mail; t=1488278992; bh=0zohcgo8gZ0/vDSvu1H6x1VPlQjggPE5X6zBapo7Au8=; h=From:Subject:Date:References:To:In-Reply-To:From; b=qXrPaaeVdYI+naXGL6drJkBWyO2JRfGmIJ1VgSBGVfeUYiXoaifhyXpuuHg7ANFCr TtEZXCYQNsNqBToaUwUtEUZxeaIdpnhyHdVceDJApQkhZJL5Kc3lKS7z9YukrozGNv XLe5q54i95pIdZG5FesNqQV768XKz6FmNTtrPmLY= From: Stefan Eissing Content-Type: multipart/mixed; boundary="Apple-Mail=_36DD39E4-E88A-4BEA-B85F-9FF81037B9B1" Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: Re: release v1.9.2 Date: Tue, 28 Feb 2017 11:49:55 +0100 References: <216BB26F-BEFF-4631-B961-B89755EB4283@greenbytes.de> <05F72C34-0C7F-4613-83E7-5BA5058B4E45@greenbytes.de> <8538ded9-4409-459e-306e-e233580d7a94@profihost.ag> <1B716C89-C970-473F-BCF3-173380CC0514@greenbytes.de> <085AD8B5-4D39-4F79-B1C6-EB11495045D6@greenbytes.de> <326410d2-087c-4a76-e3e0-c5b8a858fd2a@profihost.ag> <14B1D933-CA63-411B-8C32-BD963C4B6F11@greenbytes.de> To: dev@httpd.apache.org In-Reply-To: Message-Id: <210D7FD1-FE97-439F-8A6B-12E06923CF26@greenbytes.de> X-Mailer: Apple Mail (2.3259) archived-at: Tue, 28 Feb 2017 10:50:07 -0000 --Apple-Mail=_36DD39E4-E88A-4BEA-B85F-9FF81037B9B1 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Meh, my mistake. Sorry about that. Did not cleanup properly. Dare you = with an improved version: --Apple-Mail=_36DD39E4-E88A-4BEA-B85F-9FF81037B9B1 Content-Disposition: attachment; filename=h2_192_alloc_mutex_v2.diff Content-Type: application/octet-stream; x-unix-mode=0644; name="h2_192_alloc_mutex_v2.diff" Content-Transfer-Encoding: 7bit Index: modules/http2/h2_session.c =================================================================== --- modules/http2/h2_session.c (revision 1784618) +++ modules/http2/h2_session.c (working copy) @@ -702,6 +702,12 @@ ap_assert(session->ngh2); nghttp2_session_del(session->ngh2); session->ngh2 = NULL; + + if (session->allocator_mutex) { + apr_allocator_t *allocator = apr_pool_allocator_get(session->c->pool); + apr_allocator_mutex_set(allocator, NULL); + session->allocator_mutex = NULL; + } return APR_SUCCESS; } @@ -718,125 +724,136 @@ uint32_t n; apr_pool_t *pool = NULL; h2_session *session; + apr_status_t status; + int rv; - apr_status_t status = apr_allocator_create(&allocator); - if (status != APR_SUCCESS) { - return NULL; - } - apr_allocator_max_free_set(allocator, ap_max_mem_free); - apr_pool_create_ex(&pool, c->pool, NULL, allocator); + apr_pool_create(&pool, c->pool); if (!pool) { - apr_allocator_destroy(allocator); return NULL; } apr_pool_tag(pool, "h2_session"); - apr_allocator_owner_set(allocator, pool); - status = apr_thread_mutex_create(&mutex, APR_THREAD_MUTEX_DEFAULT, pool); - if (status != APR_SUCCESS) { + + /* get h2_session a lifetime beyond its pool and everything + * connected to it. */ + session = apr_pcalloc(pool, sizeof(h2_session)); + if (!session) { apr_pool_destroy(pool); return NULL; } - apr_allocator_mutex_set(allocator, mutex); - /* get h2_session a lifetime beyond its pool and everything - * connected to it. */ - session = apr_pcalloc(pool, sizeof(h2_session)); - if (session) { - int rv; - session->id = c->id; - session->c = c; - session->r = r; - session->s = h2_ctx_server_get(ctx); - session->pool = pool; - session->config = h2_config_sget(session->s); - session->workers = workers; - - session->state = H2_SESSION_ST_INIT; - session->local.accepting = 1; - session->remote.accepting = 1; - - apr_pool_pre_cleanup_register(pool, session, session_pool_cleanup); - - session->max_stream_count = h2_config_geti(session->config, - H2_CONF_MAX_STREAMS); - session->max_stream_mem = h2_config_geti(session->config, - H2_CONF_STREAM_MAX_MEM); - - status = apr_thread_cond_create(&session->iowait, session->pool); + session->id = c->id; + session->c = c; + session->r = r; + session->s = h2_ctx_server_get(ctx); + session->pool = pool; + session->config = h2_config_sget(session->s); + session->workers = workers; + + session->state = H2_SESSION_ST_INIT; + session->local.accepting = 1; + session->remote.accepting = 1; + + apr_pool_pre_cleanup_register(pool, session, session_pool_cleanup); + /* normally, connection pool allocators work without a mutex. However + * for http/2 connections, more multi-threading is going on and a mutex + * seems to be necessary. Set one, if it is missing. */ + allocator = apr_pool_allocator_get(c->pool); + mutex = apr_allocator_mutex_get(allocator); + if (mutex == NULL) { + status = apr_thread_mutex_create(&mutex, APR_THREAD_MUTEX_DEFAULT, pool); if (status != APR_SUCCESS) { + apr_pool_destroy(pool); return NULL; } - - session->monitor = apr_pcalloc(pool, sizeof(h2_stream_monitor)); - if (session->monitor == NULL) { - return NULL; - } - session->monitor->ctx = session; - session->monitor->on_state_enter = on_stream_state_enter; - session->monitor->on_state_event = on_stream_state_event; - - session->mplx = h2_mplx_create(c, session->pool, session->config, - session->s->timeout, workers); - - h2_mplx_set_consumed_cb(session->mplx, update_window, session); - - /* Install the connection input filter that feeds the session */ - session->cin = h2_filter_cin_create(session->pool, - h2_session_receive, session); - ap_add_input_filter("H2_IN", session->cin, r, c); - - h2_conn_io_init(&session->io, c, session->config); - session->bbtmp = apr_brigade_create(session->pool, c->bucket_alloc); - - status = init_callbacks(c, &callbacks); - if (status != APR_SUCCESS) { - ap_log_cerror(APLOG_MARK, APLOG_ERR, status, c, APLOGNO(02927) - "nghttp2: error in init_callbacks"); - return NULL; - } - - rv = nghttp2_option_new(&options); - if (rv != 0) { - ap_log_cerror(APLOG_MARK, APLOG_ERR, APR_EGENERAL, c, - APLOGNO(02928) "nghttp2_option_new: %s", - nghttp2_strerror(rv)); - return NULL; - } - nghttp2_option_set_peer_max_concurrent_streams( - options, (uint32_t)session->max_stream_count); - /* We need to handle window updates ourself, otherwise we - * get flooded by nghttp2. */ - nghttp2_option_set_no_auto_window_update(options, 1); - - rv = nghttp2_session_server_new2(&session->ngh2, callbacks, - session, options); - nghttp2_session_callbacks_del(callbacks); - nghttp2_option_del(options); - - if (rv != 0) { - ap_log_cerror(APLOG_MARK, APLOG_ERR, APR_EGENERAL, c, - APLOGNO(02929) "nghttp2_session_server_new: %s", - nghttp2_strerror(rv)); - return NULL; - } - - n = h2_config_geti(session->config, H2_CONF_PUSH_DIARY_SIZE); - session->push_diary = h2_push_diary_create(session->pool, n); - - if (APLOGcdebug(c)) { - ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, c, - H2_SSSN_LOG(APLOGNO(03200), session, - "created, max_streams=%d, stream_mem=%d, " - "workers_limit=%d, workers_max=%d, " - "push_diary(type=%d,N=%d)"), - (int)session->max_stream_count, - (int)session->max_stream_mem, - session->mplx->workers_limit, - session->mplx->workers_max, - session->push_diary->dtype, - (int)session->push_diary->N); - } + apr_allocator_mutex_set(allocator, mutex); + /* need to remember unsetting on pool clearance */ + session->allocator_mutex = mutex; } + + session->max_stream_count = h2_config_geti(session->config, + H2_CONF_MAX_STREAMS); + session->max_stream_mem = h2_config_geti(session->config, + H2_CONF_STREAM_MAX_MEM); + + status = apr_thread_cond_create(&session->iowait, session->pool); + if (status != APR_SUCCESS) { + apr_pool_destroy(pool); + return NULL; + } + + session->monitor = apr_pcalloc(pool, sizeof(h2_stream_monitor)); + if (session->monitor == NULL) { + apr_pool_destroy(pool); + return NULL; + } + session->monitor->ctx = session; + session->monitor->on_state_enter = on_stream_state_enter; + session->monitor->on_state_event = on_stream_state_event; + + session->mplx = h2_mplx_create(c, session->pool, session->config, + session->s->timeout, workers); + + h2_mplx_set_consumed_cb(session->mplx, update_window, session); + + /* Install the connection input filter that feeds the session */ + session->cin = h2_filter_cin_create(session->pool, + h2_session_receive, session); + ap_add_input_filter("H2_IN", session->cin, r, c); + + h2_conn_io_init(&session->io, c, session->config); + session->bbtmp = apr_brigade_create(session->pool, c->bucket_alloc); + + status = init_callbacks(c, &callbacks); + if (status != APR_SUCCESS) { + ap_log_cerror(APLOG_MARK, APLOG_ERR, status, c, APLOGNO(02927) + "nghttp2: error in init_callbacks"); + apr_pool_destroy(pool); + return NULL; + } + + rv = nghttp2_option_new(&options); + if (rv != 0) { + ap_log_cerror(APLOG_MARK, APLOG_ERR, APR_EGENERAL, c, + APLOGNO(02928) "nghttp2_option_new: %s", + nghttp2_strerror(rv)); + apr_pool_destroy(pool); + return NULL; + } + nghttp2_option_set_peer_max_concurrent_streams( + options, (uint32_t)session->max_stream_count); + /* We need to handle window updates ourself, otherwise we + * get flooded by nghttp2. */ + nghttp2_option_set_no_auto_window_update(options, 1); + + rv = nghttp2_session_server_new2(&session->ngh2, callbacks, + session, options); + nghttp2_session_callbacks_del(callbacks); + nghttp2_option_del(options); + + if (rv != 0) { + ap_log_cerror(APLOG_MARK, APLOG_ERR, APR_EGENERAL, c, + APLOGNO(02929) "nghttp2_session_server_new: %s", + nghttp2_strerror(rv)); + apr_pool_destroy(pool); + return NULL; + } + + n = h2_config_geti(session->config, H2_CONF_PUSH_DIARY_SIZE); + session->push_diary = h2_push_diary_create(session->pool, n); + + if (APLOGcdebug(c)) { + ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, c, + H2_SSSN_LOG(APLOGNO(03200), session, + "created, max_streams=%d, stream_mem=%d, " + "workers_limit=%d, workers_max=%d, " + "push_diary(type=%d,N=%d)"), + (int)session->max_stream_count, + (int)session->max_stream_mem, + session->mplx->workers_limit, + session->mplx->workers_max, + session->push_diary->dtype, + (int)session->push_diary->N); + } return session; } Index: modules/http2/h2_session.h =================================================================== --- modules/http2/h2_session.h (revision 1784618) +++ modules/http2/h2_session.h (working copy) @@ -121,6 +121,7 @@ apr_bucket_brigade *bbtmp; /* brigade for keeping temporary data */ struct apr_thread_cond_t *iowait; /* our cond when trywaiting for data */ + struct apr_thread_mutex_t *allocator_mutex; /* c->pool->allocator mutex if we created one */ char status[64]; /* status message for scoreboard */ int last_status_code; /* the one already reported */ --Apple-Mail=_36DD39E4-E88A-4BEA-B85F-9FF81037B9B1 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > Am 28.02.2017 um 08:41 schrieb Stefan Priebe - Profihost AG = : >=20 > That one breaks everyting. Multiple crashes per second. >=20 > [Thread debugging using libthread_db enabled] > Using host libthread_db library = "/lib/x86_64-linux-gnu/libthread_db.so.1". > Core was generated by `/usr/local/apache/bin/httpd -DFOREGROUND'. > Program terminated with signal SIGABRT, Aborted. > #0 0x00007f02724ea067 in raise () from = /lib/x86_64-linux-gnu/libc.so.6 > #0 0x00007f02724ea067 in raise () from = /lib/x86_64-linux-gnu/libc.so.6 > #1 0x00007f02724eb448 in abort () from = /lib/x86_64-linux-gnu/libc.so.6 > #2 0x00007f02724e3266 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #3 0x00007f02724e3312 in __assert_fail () from > /lib/x86_64-linux-gnu/libc.so.6 > #4 0x00007f027287062f in __pthread_tpp_change_priority () > from /lib/x86_64-linux-gnu/libpthread.so.0 > #5 0x00007f0272865e9f in __pthread_mutex_lock_full () > from /lib/x86_64-linux-gnu/libpthread.so.0 > #6 0x00007f0272a98db8 in allocator_alloc (in_size=3Din_size@entry=3D803= 2, > allocator=3D0x7f025c03f3c0) at memory/unix/apr_pools.c:244 > #7 apr_allocator_alloc (allocator=3D0x7f025c03f3c0, = size=3Dsize@entry=3D8032) > at memory/unix/apr_pools.c:438 > #8 0x00007f0272cbcac4 in apr_bucket_alloc (size=3D8032, = size@entry=3D8000, > list=3D0x7f022c0008e8) at buckets/apr_buckets_alloc.c:157 > #9 0x00007f0272cbdca2 in socket_bucket_read (a=3D0x7f022c000b08, > str=3D0x7f02627a87b8, len=3D0x7f02627a87c0, block=3D) > at buckets/apr_buckets_socket.c:34 > #10 0x0000565098cf1fb1 in ap_core_input_filter (f=3D0x7f022c0056b0, > b=3D0x7f022c005630, mode=3D, = block=3DAPR_NONBLOCK_READ, > readbytes=3D5) at core_filters.c:235 > #11 0x0000565098d3aaca in logio_in_filter (f=3D, > bb=3D0x7f022c005630, mode=3D, block=3D, > readbytes=3D) at mod_logio.c:165 > #12 0x0000565098d45b9a in bio_filter_in_read (bio=3D0x7f024800b460, > in=3D0x7f02480492a3 "", inlen=3D5) at ssl_engine_io.c:483 > #13 0x00007f02736b014c in BIO_read () > from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 > #14 0x00007f0273a13c92 in ?? () from > /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 > #15 0x00007f0273a1548d in ?? () from > /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 > #16 0x00007f0273a12024 in ?? () from > /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 > #17 0x0000565098d469f3 in ssl_io_input_read (inctx=3D0x7f022c0035b8, > buf=3D0x7f022c003600 "GET > /media/image/07/3c/c2/4612_13721_160192280322.jpeg HTTP/1.1\r\nHost: > www.onlinestore.it\r\nUser-Agent: curl/7.15+ (x64-criteo) = libcurl/7.15+ > OpenSSL zlib libidn\r\nAccept: */*\r\n\r\n", len=3D0x7f02627a8b38) > at ssl_engine_io.c:614 > #18 0x0000565098d493ec in ssl_io_filter_input (f=3D0x7f022c005608, > bb=3D0x7f022c006dd0, mode=3D, block=3D, > readbytes=3D8000) at ssl_engine_io.c:1474 > #19 0x0000565098d5cd85 in h2_filter_core_input (f=3D0x7f022c005980, > brigade=3D0x7f022c006dd0, mode=3D738225624, = block=3DAPR_NONBLOCK_READ, > readbytes=3D8000) at h2_filter.c:149 > #20 0x0000565098d6809b in h2_session_read (session=3D0x7f022c006920, > block=3D) at h2_session.c:1440 > #21 0x0000565098d6b97f in h2_session_process (session=3D0x7f022c006920, > async=3D3964) at h2_session.c:2074 > #22 0x0000565098d5b84e in h2_conn_run (ctx=3D, > c=3D0x7f025c03f7d8) > at h2_conn.c:218 > #23 0x0000565098d5e551 in h2_h2_process_conn (c=3D0x7f025c03f7d8) at > h2_h2.c:658 > #24 0x0000565098d00fd0 in ap_run_process_connection (c=3D0x7f025c03f7d8)= > at connection.c:42 > #25 0x0000565098d9b6d0 in process_socket (my_thread_num=3D, > my_child_num=3D, cs=3D0x7f025c03f748, = sock=3D, > p=3D, thd=3D) at event.c:1134 > #26 worker_thread (thd=3D0xf5e, dummy=3D0xf7c) at event.c:2137 > #27 0x00007f02728680a4 in start_thread () > from /lib/x86_64-linux-gnu/libpthread.so.0 > #28 0x00007f027259d62d in clone () from = /lib/x86_64-linux-gnu/libc.so.6 > (gdb) (gdb) quit > Reading symbols from /usr/local/apache/bin/httpd...Reading symbols = from > /usr/lib/debug//usr/local/apache2/bin/httpd...done. > done. > [Thread debugging using libthread_db enabled] > Using host libthread_db library = "/lib/x86_64-linux-gnu/libthread_db.so.1". > Core was generated by `/usr/local/apache/bin/httpd -DFOREGROUND'. > Program terminated with signal SIGABRT, Aborted. > #0 0x00007f02724ea067 in raise () from = /lib/x86_64-linux-gnu/libc.so.6 > #0 0x00007f02724ea067 in raise () from = /lib/x86_64-linux-gnu/libc.so.6 > #1 0x00007f02724eb448 in abort () from = /lib/x86_64-linux-gnu/libc.so.6 > #2 0x00007f02724e3266 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #3 0x00007f02724e3312 in __assert_fail () from > /lib/x86_64-linux-gnu/libc.so.6 > #4 0x00007f027287062f in __pthread_tpp_change_priority () > from /lib/x86_64-linux-gnu/libpthread.so.0 > #5 0x00007f0272865e9f in __pthread_mutex_lock_full () > from /lib/x86_64-linux-gnu/libpthread.so.0 > #6 0x00007f0272a98db8 in allocator_alloc (in_size=3Din_size@entry=3D803= 2, > allocator=3D0x7f025c03d320) at memory/unix/apr_pools.c:244 > #7 apr_allocator_alloc (allocator=3D0x7f025c03d320, = size=3Dsize@entry=3D8032) > at memory/unix/apr_pools.c:438 > #8 0x00007f0272cbcac4 in apr_bucket_alloc (size=3D8032, = size@entry=3D8000, > list=3D0x7f02340008e8) at buckets/apr_buckets_alloc.c:157 > #9 0x00007f0272cbdca2 in socket_bucket_read (a=3D0x7f0234000b08, > str=3D0x7f0260fa57b8, len=3D0x7f0260fa57c0, block=3D) > at buckets/apr_buckets_socket.c:34 > #10 0x0000565098cf1fb1 in ap_core_input_filter (f=3D0x7f02340056b0, > b=3D0x7f0234005630, mode=3D, = block=3DAPR_NONBLOCK_READ, > readbytes=3D5) at core_filters.c:235 > #11 0x0000565098d3aaca in logio_in_filter (f=3D, > bb=3D0x7f0234005630, mode=3D, block=3D, > readbytes=3D) at mod_logio.c:165 > #12 0x0000565098d45b9a in bio_filter_in_read (bio=3D0x7f022801ff50, > in=3D0x7f02280345d3 "(\002\177", inlen=3D5) at ssl_engine_io.c:483 > #13 0x00007f02736b014c in BIO_read () > from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 > #14 0x00007f0273a13c92 in ?? () from > /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 > #15 0x00007f0273a1548d in ?? () from > /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 > #16 0x00007f0273a12024 in ?? () from > /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 > #17 0x0000565098d469f3 in ssl_io_input_read (inctx=3D0x7f02340035b8, > buf=3D0x7f0234003600 "", len=3D0x7f0260fa5b38) at = ssl_engine_io.c:614 > #18 0x0000565098d493ec in ssl_io_filter_input (f=3D0x7f0234005608, > bb=3D0x7f023400cd00, mode=3D, block=3D, > readbytes=3D8000) at ssl_engine_io.c:1474 > #19 0x0000565098d5cd85 in h2_filter_core_input (f=3D0x7f0234005980, > brigade=3D0x7f023400cd00, mode=3D872467720, = block=3DAPR_NONBLOCK_READ, > readbytes=3D8000) at h2_filter.c:149 > #20 0x0000565098d6809b in h2_session_read (session=3D0x7f023400c850, > block=3D) at h2_session.c:1440 > #21 0x0000565098d6b97f in h2_session_process (session=3D0x7f023400c850, > async=3D4029) at h2_session.c:2074 > #22 0x0000565098d5b84e in h2_conn_run (ctx=3D, > c=3D0x7f025c03d738) > at h2_conn.c:218 > #23 0x0000565098d5e551 in h2_h2_process_conn (c=3D0x7f025c03d738) at > h2_h2.c:658 > #24 0x0000565098d00fd0 in ap_run_process_connection (c=3D0x7f025c03d738)= > at connection.c:42 > #25 0x0000565098d9b6d0 in process_socket (my_thread_num=3D, > my_child_num=3D, cs=3D0x7f025c03d6a8, = sock=3D, > p=3D, thd=3D) at event.c:1134 > #26 worker_thread (thd=3D0xf9c, dummy=3D0xfbd) at event.c:2137 > #27 0x00007f02728680a4 in start_thread () > from /lib/x86_64-linux-gnu/libpthread.so.0 > #28 0x00007f027259d62d in clone () from = /lib/x86_64-linux-gnu/libc.so.6 >=20 > Stefan >=20 > Am 27.02.2017 um 19:44 schrieb Stefan Eissing: >> Damnit. Can you try the attached patch on top of mod_http2 v1.9.2? = This one sets a mutex on the main connection allocator if missing and is = pretty close to the version we ran successfully with on your site for = days. >>=20 >> Thanks again! >>=20 >> -Stefan >>=20 >>=20 >>=20 >>=20 >>> Am 27.02.2017 um 16:22 schrieb Stefan Priebe - Profihost AG = : >>>=20 >>> Hi Stefan, >>>=20 >>> two new crashes here. >>>=20 >>> Reading symbols from /usr/local/apache/bin/httpd...Reading symbols = from >>> /usr/lib/debug//usr/local/apache2/bin/httpd...done. >>> done. >>> (gdb) (gdb) quit >>> Reading symbols from /usr/local/apache/bin/httpd...Reading symbols = from >>> /usr/lib/debug//usr/local/apache2/bin/httpd...done. >>> done. >>> [Thread debugging using libthread_db enabled] >>> Using host libthread_db library = "/lib/x86_64-linux-gnu/libthread_db.so.1". >>> Core was generated by `/usr/local/apache/bin/httpd -DFOREGROUND'. >>> Program terminated with signal SIGSEGV, Segmentation fault. >>> #0 allocator_free (node=3D0x0, allocator=3D0x7f458005a320) >>> at memory/unix/apr_pools.c:381 >>> #0 allocator_free (node=3D0x0, allocator=3D0x7f458005a320) >>> at memory/unix/apr_pools.c:381 >>> #1 apr_pool_clear (pool=3D0x7f458004bec8) at = memory/unix/apr_pools.c:793 >>> #2 0x0000560ce83e5718 in ap_push_pool (queue_info=3D0x0, >>> pool_to_recycle=3D0x7f458005a328) at fdqueue.c:234 >>> #3 0x0000560ce83e0a5a in process_lingering_close = (cs=3D0x7f458004c158, >>> pfd=3D0x560ce8b8dda8) at event.c:1513 >>> #4 0x0000560ce83e4590 in listener_thread (thd=3D0x0, = dummy=3D0x5497f5242585c) >>> at event.c:1837 >>> #5 0x00007f45967610a4 in start_thread () >>> from /lib/x86_64-linux-gnu/libpthread.so.0 >>> #6 0x00007f459649662d in clone () from = /lib/x86_64-linux-gnu/libc.so.6 >>> (gdb) (gdb) quit >>>=20 >>>=20 >>> Reading symbols from /usr/local/apache/bin/httpd...Reading symbols = from >>> /usr/lib/debug//usr/local/apache2/bin/httpd...done. >>> done. >>> [Thread debugging using libthread_db enabled] >>> Using host libthread_db library = "/lib/x86_64-linux-gnu/libthread_db.so.1". >>> Core was generated by `/usr/local/apache/bin/httpd -DFOREGROUND'. >>> Program terminated with signal SIGSEGV, Segmentation fault. >>> #0 allocator_free (node=3D0x0, allocator=3D0x7f4580062840) >>> at memory/unix/apr_pools.c:381 >>> #0 allocator_free (node=3D0x0, allocator=3D0x7f4580062840) >>> at memory/unix/apr_pools.c:381 >>> #1 apr_pool_clear (pool=3D0x7f4580069188) at = memory/unix/apr_pools.c:793 >>> #2 0x0000560ce83e5718 in ap_push_pool (queue_info=3D0x0, >>> pool_to_recycle=3D0x7f4580062848) at fdqueue.c:234 >>> #3 0x0000560ce83e0a5a in process_lingering_close = (cs=3D0x7f4580069418, >>> pfd=3D0x560ce8b8dda8) at event.c:1513 >>> #4 0x0000560ce83e4590 in listener_thread (thd=3D0x0, = dummy=3D0x549845279ce62) >>> at event.c:1837 >>> #5 0x00007f45967610a4 in start_thread () >>> from /lib/x86_64-linux-gnu/libpthread.so.0 >>> #6 0x00007f459649662d in clone () from = /lib/x86_64-linux-gnu/libc.so.6 >>> (gdb) (gdb) quit >>>=20 >>> Stefan >>> Am 26.02.2017 um 19:46 schrieb Stefan Priebe - Profihost AG: >>>> Hi Stefan, >>>>=20 >>>> currently everything fine. No segfaults. >>>>=20 >>>> Stefan >>>>=20 >>>> Am 25.02.2017 um 20:40 schrieb Stefan Priebe - Profihost AG: >>>>> Hi Stefan, >>>>> Am 25.02.2017 um 13:51 schrieb Stefan Eissing: >>>>>> Stefan, >>>>>>=20 >>>>>> whenever you have time, please deploy = https://github.com/icing/mod_h2/releases/tag/v1.9.2 >>>>>>=20 >>>>>> I added an own allocator with mutex to the http/2 main session. = That is something of a middle-ground between placing that on the main = conn (as we had in the crash free version) and 1.9.1 behaviour. Thanks! >>>>>=20 >>>>> done. But please keep in mind that this crash might be very rare = and >>>>> might even have happened with v1.9.0 if we've waited more time. >>>>>=20 >>>>> Greets, >>>>> Stefan >>>>>=20 >>>>>> -Stefan >>>>>>=20 >>>>>>> Am 24.02.2017 um 19:31 schrieb Stefan Priebe - Profihost AG = : >>>>>>>=20 >>>>>>> Hi Yann, >>>>>>>=20 >>>>>>> here we go: >>>>>>>=20 >>>>>>> (gdb) p *ipsub >>>>>>> $1 =3D {family =3D 2, sub =3D {1367753145, 0, 0, 0}, mask =3D = {4294967295, >>>>>>> 4294967295, 4294967295, 4294967295}} >>>>>>>=20 >>>>>>> (gdb) p *sa >>>>>>> $2 =3D {pool =3D 0x10b500c4ff0b0a, hostname =3D = 0x503040203030102 >>>>>> Cannot access memory at address 0x503040203030102>, >>>>>>> servname =3D 0x17d010000040405 >>>>>> 0x17d010000040405>, port =3D 770, family =3D 554829073, >>>>>>> salen =3D 319177009, ipaddr_len =3D 570909009, addr_str_len =3D = -2127424399, >>>>>>> ipaddr_ptr =3D 0x24f0d15215c1b142, >>>>>>> next =3D 0x17160a0982726233, sa =3D {sin =3D {sin_family =3D = 6424, sin_port =3D >>>>>>> 9498, sin_addr =3D {s_addr =3D 690497318}, >>>>>>> sin_zero =3D "*456789:"}, sin6 =3D {sin6_family =3D 6424, = sin6_port =3D >>>>>>> 9498, sin6_flowinfo =3D 690497318, sin6_addr =3D {__in6_u =3D { >>>>>>> __u6_addr8 =3D "*456789:CDEFGHIJ", __u6_addr16 =3D = {13354, 13877, >>>>>>> 14391, 14905, 17475, 17989, 18503, 19017}, __u6_addr32 =3D { >>>>>>> 909456426, 976828471, 1178944579, 1246316615}}}, >>>>>>> sin6_scope_id =3D 1448432723}, sas =3D {ss_family =3D 6424, >>>>>>> __ss_align =3D 4195446337656140842, >>>>>>> __ss_padding =3D >>>>>>> = "CDEFGHIJSTUVWXYZcdefghijstuvwxyz\203\204\205\206\207\210\211\212\222\223\= 224\225\226\227\230\231\232\242\243\244\245\246\247\250\251\252\262\263\26= 4\265\266\267\270\271\272\302\303\304\305\306\307\310\311\312\322\323\324\= 325\326\327\330\331\332\341\342\343\344\345\346\347\350\351\352\361\362\36= 3\364\365\366\367\370\371\372\377\304\000\037\001\000\003"}}} >>>>>>>=20 >>>>>>> (gdb) p *(struct in6_addr *)sa >>>>>>> $3 =3D {__in6_u =3D {__u6_addr8 =3D >>>>>>> "\n\v\377\304\000\265\020\000\002\001\003\003\002\004\003\005", >>>>>>> __u6_addr16 =3D {2826, 50431, 46336, >>>>>>> 16, 258, 771, 1026, 1283}, __u6_addr32 =3D {3305048842, = 1094912, >>>>>>> 50528514, 84083714}}} >>>>>>>=20 >>>>>>>=20 >>>>>>> Stefan >>>>>>>=20 >>>>>>> Am 24.02.2017 um 14:18 schrieb Yann Ylavic: >>>>>>>> Hi Stefan (Priebe), >>>>>>>>=20 >>>>>>>> Is IPv6 (really) involved in your network? >>>>>>>>=20 >>>>>>>> Could you please show up the gdb output of the below ? >>>>>>>>=20 >>>>>>>> On Fri, Feb 24, 2017 at 2:07 PM, Yann Ylavic = wrote: >>>>>>>>>=20 >>>>>>>>> 1078 APR_DECLARE(int) apr_ipsubnet_test(apr_ipsubnet_t *ipsub, >>>>>>>>> apr_sockaddr_t *sa) >>>>>>>>> 1079 { >>>>>>>>> 1080 #if APR_HAVE_IPV6 >>>>>>>>> 1081 /* XXX This line will segv on Win32 build with = APR_HAVE_IPV6, >>>>>>>>> 1082 * but without the IPV6 drivers installed. >>>>>>>>> 1083 */ >>>>>>>>> 1084 if (sa->family =3D=3D AF_INET) { >>>>>>>>> 1085 if (ipsub->family =3D=3D AF_INET && >>>>>>>>> 1086 ((sa->sa.sin.sin_addr.s_addr & = ipsub->mask[0]) =3D=3D >>>>>>>>> ipsub->sub[0])) { >>>>>>>>> 1087 return 1; >>>>>>>>> 1088 } >>>>>>>>> 1089 } >>>>>>>>> 1090 else if (IN6_IS_ADDR_V4MAPPED((struct in6_addr = *)sa->ipaddr_ptr)) { >>>>>>>>> 1091 if (ipsub->family =3D=3D AF_INET && >>>>>>>>> 1092 (((apr_uint32_t *)sa->ipaddr_ptr)[3] & >>>>>>>>> ipsub->mask[0]) =3D=3D ipsub->sub[0]) { >>>>>>>>> 1093 return 1; >>>>>>>>> 1094 } >>>>>>>>> 1095 } >>>>>>>>=20 >>>>>>>> (gdb) p *ipsub >>>>>>>> (gdb) p *sa >>>>>>>> (gdb) p *(struct in6_addr *)sa >>>>>>>>=20 >>>>>>>> and possibly more to come... >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> Thanks, >>>>>>>> Yann. >>>>>>>>=20 >>>>>>=20 >>>>>> Stefan Eissing >>>>>>=20 >>>>>> bytes GmbH >>>>>> Hafenstrasse 16 >>>>>> 48155 M=C3=BCnster >>>>>> www.greenbytes.de >>>>>>=20 >>=20 >> Stefan Eissing >>=20 >> bytes GmbH >> Hafenstrasse 16 >> 48155 M=C3=BCnster >> www.greenbytes.de >>=20 Stefan Eissing bytes GmbH Hafenstrasse 16 48155 M=C3=BCnster www.greenbytes.de --Apple-Mail=_36DD39E4-E88A-4BEA-B85F-9FF81037B9B1--