Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 34234200C32 for ; Thu, 9 Mar 2017 18:09:20 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 3293B160B75; Thu, 9 Mar 2017 17:09:20 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id CEB06160B5F for ; Thu, 9 Mar 2017 18:09:18 +0100 (CET) Received: (qmail 9359 invoked by uid 500); 9 Mar 2017 17:09:17 -0000 Mailing-List: contact dev-help@httpd.apache.org; run by ezmlm Precedence: bulk Reply-To: dev@httpd.apache.org list-help: list-unsubscribe: List-Post: List-Id: Delivered-To: mailing list dev@httpd.apache.org Received: (qmail 9349 invoked by uid 99); 9 Mar 2017 17:09:17 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Mar 2017 17:09:17 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 5A1C2C05A9 for ; Thu, 9 Mar 2017 17:09:17 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.399 X-Spam-Level: X-Spam-Status: No, score=0.399 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, KAM_NUMSUBJECT=0.5, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=greenbytes.de header.b=SDogu520; dkim=pass (1024-bit key) header.d=greenbytes.de header.b=RkqxHu0b Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id hwtDjDgwuroq for ; Thu, 9 Mar 2017 17:09:12 +0000 (UTC) Received: from mail.greenbytes.de (mail.greenbytes.de [5.10.171.186]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 547F35F647 for ; Thu, 9 Mar 2017 17:09:12 +0000 (UTC) Received: by mail.greenbytes.de (Postfix, from userid 117) id 2129015A0E68; Thu, 9 Mar 2017 18:09:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=greenbytes.de; s=mail; t=1489079345; bh=CIrf6dpUpeT1Zd4QETaAlfulqGDCkIdks/mxU32/fpw=; h=From:Subject:Date:References:To:In-Reply-To:From; b=SDogu520wPb6hskda+CE2dTGqB/QnBMCgk3azxeRKGXcS3B8KNU2DKWvLKgmutDdg FHJabI3MRXVaR45RgI1U23ONjgeKxRJiYk23xXunUX0+j3zmPIQB1TYq8TnWEWG2vO Q+i7yhOcUng0cdO1707t82uF9n7dzxXgiyAQZZnA= Received: from [192.168.1.38] (unknown [192.168.1.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mail.greenbytes.de (Postfix) with ESMTPSA id 3431915A05F9 for ; Thu, 9 Mar 2017 18:09:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=greenbytes.de; s=mail; t=1489079344; bh=CIrf6dpUpeT1Zd4QETaAlfulqGDCkIdks/mxU32/fpw=; h=From:Subject:Date:References:To:In-Reply-To:From; b=RkqxHu0byjaalF83IdVsPQtfZwAyuWXzCWwFndSwnbBXJ0hz4c5BZE7HxxsG2bD5G GnjBm8otzeGGRDWCoQVvKqkc7LYX5kpFlDBbSTx0yJ3/AhiICDBmSVrDMxtZqB4OmC 1FyjXHFY4DcegCnnvlnslanBgqoGp9B9dBXtT9i0= From: Stefan Eissing Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: Re: release v1.9.2 Date: Thu, 9 Mar 2017 18:09:03 +0100 References: <216BB26F-BEFF-4631-B961-B89755EB4283@greenbytes.de> <8538ded9-4409-459e-306e-e233580d7a94@profihost.ag> <1B716C89-C970-473F-BCF3-173380CC0514@greenbytes.de> <085AD8B5-4D39-4F79-B1C6-EB11495045D6@greenbytes.de> <326410d2-087c-4a76-e3e0-c5b8a858fd2a@profihost.ag> <14B1D933-CA63-411B-8C32-BD963C4B6F11@greenbytes.de> <210D7FD1-FE97-439F-8A6B-12E06923CF26@greenbytes.de> <350d0bcd-63fd-1ac0-536c-8fa0fff0c9a4@profihost.ag> <9a6ee2b2-1005-2d23-5920-890510c9457a@profihos t.ag> <078F5037-91B7-433B-B61F-4D1AE9D7DF83@greenbytes.de> <1017f615-4cae-b40e-ab92-0f2e4fe89fa3@profihost.ag> To: dev@httpd.apache.org In-Reply-To: Message-Id: <47F2D9AE-7477-4F84-9346-892E94852C33@greenbytes.de> X-Mailer: Apple Mail (2.3259) archived-at: Thu, 09 Mar 2017 17:09:20 -0000 Thanks for running our patches with many changes! Not a mistake to have = it running fine for four days. ;-) Hmm, so we hunt a rare beast. After much effort from all sides we made = it rare. So, nothing to be ashamed of. Need to keep looking... > Am 09.03.2017 um 17:32 schrieb Stefan Priebe - Profihost AG = : >=20 > Hi Stefan, >=20 > while doing longer testing i can say that also the version which was > working for 4 days segfaults. So it was never solved ;-( Sorry about > that. Testing was just too short. >=20 > Greets, > Stefan >=20 > Am 05.03.2017 um 14:49 schrieb Stefan Priebe - Profihost AG: >> Hi Stefan, >>=20 >> yes this seems to correct BUT i'm not sure i the test was long enough >> where i hadn't a segfault. I'll rerun a test with that version. To >> verify this was no just a "fault". >>=20 >> Greets, >> Stefan >> Am 05.03.2017 um 14:34 schrieb Stefan Eissing: >>> Thanks. I try to summarize: >>>=20 >>> - we did see this rarely with unpatched v1.9.2 and Yann's mpm patch = v?? >>> - we still see this rarely with the patch adding mutex to the main = conn allocator (so this seems not to change anything) >>> - we did not see this with v1.9.2 and Yann's patch on ptrans version = ?? >>>=20 >>> Is this correct? >>>=20 >>>> Am 04.03.2017 um 22:34 schrieb Stefan Priebe - Profihost AG = : >>>>=20 >>>> Hi Stefan, >>>>=20 >>>> current trace - not sure whether this is http2 related at all: >>>>=20 >>>> Core was generated by `/usr/local/apache/bin/httpd -DFOREGROUND'. >>>> Program terminated with signal SIGSEGV, Segmentation fault. >>>> #0 allocator_free (node=3D0x0, allocator=3D0x7fbf4c05a410) >>>> at memory/unix/apr_pools.c:381 >>>> #0 allocator_free (node=3D0x0, allocator=3D0x7fbf4c05a410) >>>> at memory/unix/apr_pools.c:381 >>>> #1 apr_pool_clear (pool=3D0x7fbf4c04f888) at = memory/unix/apr_pools.c:793 >>>> #2 0x00005642878c2818 in ap_push_pool (queue_info=3D0x0, >>>> pool_to_recycle=3D0x7fbf4c05a418) at fdqueue.c:234 >>>> #3 0x00005642878bdb5a in process_lingering_close = (cs=3D0x7fbf4c04fb18, >>>> pfd=3D0x5642886c7078) at event.c:1513 >>>> #4 0x00005642878c1690 in listener_thread (thd=3D0x0, = dummy=3D0x549eb52311a6f) >>>> at event.c:1837 >>>> #5 0x00007fbf5f2940a4 in start_thread () >>>> from /lib/x86_64-linux-gnu/libpthread.so.0 >>>> #6 0x00007fbf5efc962d in clone () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>=20 >>>> Stefan >>>>=20 >>>>> Am 03.03.2017 um 21:02 schrieb Stefan Priebe - Profihost AG: >>>>> Hi Stefan, >>>>>=20 >>>>> live. Sorry for the late reply. >>>>>=20 >>>>> Stefan >>>>>=20 >>>>>> Am 28.02.2017 um 11:49 schrieb Stefan Eissing: >>>>>> Meh, my mistake. Sorry about that. Did not cleanup properly. Dare = you with an improved version: >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>>> Am 28.02.2017 um 08:41 schrieb Stefan Priebe - Profihost AG = : >>>>>>>=20 >>>>>>> That one breaks everyting. Multiple crashes per second. >>>>>>>=20 >>>>>>> [Thread debugging using libthread_db enabled] >>>>>>> Using host libthread_db library = "/lib/x86_64-linux-gnu/libthread_db.so.1". >>>>>>> Core was generated by `/usr/local/apache/bin/httpd = -DFOREGROUND'. >>>>>>> Program terminated with signal SIGABRT, Aborted. >>>>>>> #0 0x00007f02724ea067 in raise () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>>>> #0 0x00007f02724ea067 in raise () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>>>> #1 0x00007f02724eb448 in abort () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>>>> #2 0x00007f02724e3266 in ?? () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>>>> #3 0x00007f02724e3312 in __assert_fail () from >>>>>>> /lib/x86_64-linux-gnu/libc.so.6 >>>>>>> #4 0x00007f027287062f in __pthread_tpp_change_priority () >>>>>>> from /lib/x86_64-linux-gnu/libpthread.so.0 >>>>>>> #5 0x00007f0272865e9f in __pthread_mutex_lock_full () >>>>>>> from /lib/x86_64-linux-gnu/libpthread.so.0 >>>>>>> #6 0x00007f0272a98db8 in allocator_alloc = (in_size=3Din_size@entry=3D8032, >>>>>>> allocator=3D0x7f025c03f3c0) at memory/unix/apr_pools.c:244 >>>>>>> #7 apr_allocator_alloc (allocator=3D0x7f025c03f3c0, = size=3Dsize@entry=3D8032) >>>>>>> at memory/unix/apr_pools.c:438 >>>>>>> #8 0x00007f0272cbcac4 in apr_bucket_alloc (size=3D8032, = size@entry=3D8000, >>>>>>> list=3D0x7f022c0008e8) at buckets/apr_buckets_alloc.c:157 >>>>>>> #9 0x00007f0272cbdca2 in socket_bucket_read (a=3D0x7f022c000b08, >>>>>>> str=3D0x7f02627a87b8, len=3D0x7f02627a87c0, block=3D) >>>>>>> at buckets/apr_buckets_socket.c:34 >>>>>>> #10 0x0000565098cf1fb1 in ap_core_input_filter = (f=3D0x7f022c0056b0, >>>>>>> b=3D0x7f022c005630, mode=3D, = block=3DAPR_NONBLOCK_READ, >>>>>>> readbytes=3D5) at core_filters.c:235 >>>>>>> #11 0x0000565098d3aaca in logio_in_filter (f=3D, >>>>>>> bb=3D0x7f022c005630, mode=3D, block=3D, >>>>>>> readbytes=3D) at mod_logio.c:165 >>>>>>> #12 0x0000565098d45b9a in bio_filter_in_read = (bio=3D0x7f024800b460, >>>>>>> in=3D0x7f02480492a3 "", inlen=3D5) at ssl_engine_io.c:483 >>>>>>> #13 0x00007f02736b014c in BIO_read () >>>>>>> from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 >>>>>>> #14 0x00007f0273a13c92 in ?? () from >>>>>>> /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 >>>>>>> #15 0x00007f0273a1548d in ?? () from >>>>>>> /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 >>>>>>> #16 0x00007f0273a12024 in ?? () from >>>>>>> /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 >>>>>>> #17 0x0000565098d469f3 in ssl_io_input_read = (inctx=3D0x7f022c0035b8, >>>>>>> buf=3D0x7f022c003600 "GET >>>>>>> /media/image/07/3c/c2/4612_13721_160192280322.jpeg = HTTP/1.1\r\nHost: >>>>>>> www.onlinestore.it\r\nUser-Agent: curl/7.15+ (x64-criteo) = libcurl/7.15+ >>>>>>> OpenSSL zlib libidn\r\nAccept: */*\r\n\r\n", len=3D0x7f02627a8b38)= >>>>>>> at ssl_engine_io.c:614 >>>>>>> #18 0x0000565098d493ec in ssl_io_filter_input (f=3D0x7f022c005608,= >>>>>>> bb=3D0x7f022c006dd0, mode=3D, block=3D, >>>>>>> readbytes=3D8000) at ssl_engine_io.c:1474 >>>>>>> #19 0x0000565098d5cd85 in h2_filter_core_input = (f=3D0x7f022c005980, >>>>>>> brigade=3D0x7f022c006dd0, mode=3D738225624, = block=3DAPR_NONBLOCK_READ, >>>>>>> readbytes=3D8000) at h2_filter.c:149 >>>>>>> #20 0x0000565098d6809b in h2_session_read = (session=3D0x7f022c006920, >>>>>>> block=3D) at h2_session.c:1440 >>>>>>> #21 0x0000565098d6b97f in h2_session_process = (session=3D0x7f022c006920, >>>>>>> async=3D3964) at h2_session.c:2074 >>>>>>> #22 0x0000565098d5b84e in h2_conn_run (ctx=3D, >>>>>>> c=3D0x7f025c03f7d8) >>>>>>> at h2_conn.c:218 >>>>>>> #23 0x0000565098d5e551 in h2_h2_process_conn (c=3D0x7f025c03f7d8) = at >>>>>>> h2_h2.c:658 >>>>>>> #24 0x0000565098d00fd0 in ap_run_process_connection = (c=3D0x7f025c03f7d8) >>>>>>> at connection.c:42 >>>>>>> #25 0x0000565098d9b6d0 in process_socket = (my_thread_num=3D, >>>>>>> my_child_num=3D, cs=3D0x7f025c03f748, = sock=3D, >>>>>>> p=3D, thd=3D) at event.c:1134 >>>>>>> #26 worker_thread (thd=3D0xf5e, dummy=3D0xf7c) at event.c:2137 >>>>>>> #27 0x00007f02728680a4 in start_thread () >>>>>>> from /lib/x86_64-linux-gnu/libpthread.so.0 >>>>>>> #28 0x00007f027259d62d in clone () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>>>> (gdb) (gdb) quit >>>>>>> Reading symbols from /usr/local/apache/bin/httpd...Reading = symbols from >>>>>>> /usr/lib/debug//usr/local/apache2/bin/httpd...done. >>>>>>> done. >>>>>>> [Thread debugging using libthread_db enabled] >>>>>>> Using host libthread_db library = "/lib/x86_64-linux-gnu/libthread_db.so.1". >>>>>>> Core was generated by `/usr/local/apache/bin/httpd = -DFOREGROUND'. >>>>>>> Program terminated with signal SIGABRT, Aborted. >>>>>>> #0 0x00007f02724ea067 in raise () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>>>> #0 0x00007f02724ea067 in raise () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>>>> #1 0x00007f02724eb448 in abort () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>>>> #2 0x00007f02724e3266 in ?? () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>>>> #3 0x00007f02724e3312 in __assert_fail () from >>>>>>> /lib/x86_64-linux-gnu/libc.so.6 >>>>>>> #4 0x00007f027287062f in __pthread_tpp_change_priority () >>>>>>> from /lib/x86_64-linux-gnu/libpthread.so.0 >>>>>>> #5 0x00007f0272865e9f in __pthread_mutex_lock_full () >>>>>>> from /lib/x86_64-linux-gnu/libpthread.so.0 >>>>>>> #6 0x00007f0272a98db8 in allocator_alloc = (in_size=3Din_size@entry=3D8032, >>>>>>> allocator=3D0x7f025c03d320) at memory/unix/apr_pools.c:244 >>>>>>> #7 apr_allocator_alloc (allocator=3D0x7f025c03d320, = size=3Dsize@entry=3D8032) >>>>>>> at memory/unix/apr_pools.c:438 >>>>>>> #8 0x00007f0272cbcac4 in apr_bucket_alloc (size=3D8032, = size@entry=3D8000, >>>>>>> list=3D0x7f02340008e8) at buckets/apr_buckets_alloc.c:157 >>>>>>> #9 0x00007f0272cbdca2 in socket_bucket_read (a=3D0x7f0234000b08, >>>>>>> str=3D0x7f0260fa57b8, len=3D0x7f0260fa57c0, block=3D) >>>>>>> at buckets/apr_buckets_socket.c:34 >>>>>>> #10 0x0000565098cf1fb1 in ap_core_input_filter = (f=3D0x7f02340056b0, >>>>>>> b=3D0x7f0234005630, mode=3D, = block=3DAPR_NONBLOCK_READ, >>>>>>> readbytes=3D5) at core_filters.c:235 >>>>>>> #11 0x0000565098d3aaca in logio_in_filter (f=3D, >>>>>>> bb=3D0x7f0234005630, mode=3D, block=3D, >>>>>>> readbytes=3D) at mod_logio.c:165 >>>>>>> #12 0x0000565098d45b9a in bio_filter_in_read = (bio=3D0x7f022801ff50, >>>>>>> in=3D0x7f02280345d3 "(\002\177", inlen=3D5) at = ssl_engine_io.c:483 >>>>>>> #13 0x00007f02736b014c in BIO_read () >>>>>>> from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 >>>>>>> #14 0x00007f0273a13c92 in ?? () from >>>>>>> /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 >>>>>>> #15 0x00007f0273a1548d in ?? () from >>>>>>> /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 >>>>>>> #16 0x00007f0273a12024 in ?? () from >>>>>>> /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 >>>>>>> #17 0x0000565098d469f3 in ssl_io_input_read = (inctx=3D0x7f02340035b8, >>>>>>> buf=3D0x7f0234003600 "", len=3D0x7f0260fa5b38) at = ssl_engine_io.c:614 >>>>>>> #18 0x0000565098d493ec in ssl_io_filter_input (f=3D0x7f0234005608,= >>>>>>> bb=3D0x7f023400cd00, mode=3D, block=3D, >>>>>>> readbytes=3D8000) at ssl_engine_io.c:1474 >>>>>>> #19 0x0000565098d5cd85 in h2_filter_core_input = (f=3D0x7f0234005980, >>>>>>> brigade=3D0x7f023400cd00, mode=3D872467720, = block=3DAPR_NONBLOCK_READ, >>>>>>> readbytes=3D8000) at h2_filter.c:149 >>>>>>> #20 0x0000565098d6809b in h2_session_read = (session=3D0x7f023400c850, >>>>>>> block=3D) at h2_session.c:1440 >>>>>>> #21 0x0000565098d6b97f in h2_session_process = (session=3D0x7f023400c850, >>>>>>> async=3D4029) at h2_session.c:2074 >>>>>>> #22 0x0000565098d5b84e in h2_conn_run (ctx=3D, >>>>>>> c=3D0x7f025c03d738) >>>>>>> at h2_conn.c:218 >>>>>>> #23 0x0000565098d5e551 in h2_h2_process_conn (c=3D0x7f025c03d738) = at >>>>>>> h2_h2.c:658 >>>>>>> #24 0x0000565098d00fd0 in ap_run_process_connection = (c=3D0x7f025c03d738) >>>>>>> at connection.c:42 >>>>>>> #25 0x0000565098d9b6d0 in process_socket = (my_thread_num=3D, >>>>>>> my_child_num=3D, cs=3D0x7f025c03d6a8, = sock=3D, >>>>>>> p=3D, thd=3D) at event.c:1134 >>>>>>> #26 worker_thread (thd=3D0xf9c, dummy=3D0xfbd) at event.c:2137 >>>>>>> #27 0x00007f02728680a4 in start_thread () >>>>>>> from /lib/x86_64-linux-gnu/libpthread.so.0 >>>>>>> #28 0x00007f027259d62d in clone () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>>>>=20 >>>>>>> Stefan >>>>>>>=20 >>>>>>>> Am 27.02.2017 um 19:44 schrieb Stefan Eissing: >>>>>>>> Damnit. Can you try the attached patch on top of mod_http2 = v1.9.2? This one sets a mutex on the main connection allocator if = missing and is pretty close to the version we ran successfully with on = your site for days. >>>>>>>>=20 >>>>>>>> Thanks again! >>>>>>>>=20 >>>>>>>> -Stefan >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>> Am 27.02.2017 um 16:22 schrieb Stefan Priebe - Profihost AG = : >>>>>>>>>=20 >>>>>>>>> Hi Stefan, >>>>>>>>>=20 >>>>>>>>> two new crashes here. >>>>>>>>>=20 >>>>>>>>> Reading symbols from /usr/local/apache/bin/httpd...Reading = symbols from >>>>>>>>> /usr/lib/debug//usr/local/apache2/bin/httpd...done. >>>>>>>>> done. >>>>>>>>> (gdb) (gdb) quit >>>>>>>>> Reading symbols from /usr/local/apache/bin/httpd...Reading = symbols from >>>>>>>>> /usr/lib/debug//usr/local/apache2/bin/httpd...done. >>>>>>>>> done. >>>>>>>>> [Thread debugging using libthread_db enabled] >>>>>>>>> Using host libthread_db library = "/lib/x86_64-linux-gnu/libthread_db.so.1". >>>>>>>>> Core was generated by `/usr/local/apache/bin/httpd = -DFOREGROUND'. >>>>>>>>> Program terminated with signal SIGSEGV, Segmentation fault. >>>>>>>>> #0 allocator_free (node=3D0x0, allocator=3D0x7f458005a320) >>>>>>>>> at memory/unix/apr_pools.c:381 >>>>>>>>> #0 allocator_free (node=3D0x0, allocator=3D0x7f458005a320) >>>>>>>>> at memory/unix/apr_pools.c:381 >>>>>>>>> #1 apr_pool_clear (pool=3D0x7f458004bec8) at = memory/unix/apr_pools.c:793 >>>>>>>>> #2 0x0000560ce83e5718 in ap_push_pool (queue_info=3D0x0, >>>>>>>>> pool_to_recycle=3D0x7f458005a328) at fdqueue.c:234 >>>>>>>>> #3 0x0000560ce83e0a5a in process_lingering_close = (cs=3D0x7f458004c158, >>>>>>>>> pfd=3D0x560ce8b8dda8) at event.c:1513 >>>>>>>>> #4 0x0000560ce83e4590 in listener_thread (thd=3D0x0, = dummy=3D0x5497f5242585c) >>>>>>>>> at event.c:1837 >>>>>>>>> #5 0x00007f45967610a4 in start_thread () >>>>>>>>> from /lib/x86_64-linux-gnu/libpthread.so.0 >>>>>>>>> #6 0x00007f459649662d in clone () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>>>>>> (gdb) (gdb) quit >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> Reading symbols from /usr/local/apache/bin/httpd...Reading = symbols from >>>>>>>>> /usr/lib/debug//usr/local/apache2/bin/httpd...done. >>>>>>>>> done. >>>>>>>>> [Thread debugging using libthread_db enabled] >>>>>>>>> Using host libthread_db library = "/lib/x86_64-linux-gnu/libthread_db.so.1". >>>>>>>>> Core was generated by `/usr/local/apache/bin/httpd = -DFOREGROUND'. >>>>>>>>> Program terminated with signal SIGSEGV, Segmentation fault. >>>>>>>>> #0 allocator_free (node=3D0x0, allocator=3D0x7f4580062840) >>>>>>>>> at memory/unix/apr_pools.c:381 >>>>>>>>> #0 allocator_free (node=3D0x0, allocator=3D0x7f4580062840) >>>>>>>>> at memory/unix/apr_pools.c:381 >>>>>>>>> #1 apr_pool_clear (pool=3D0x7f4580069188) at = memory/unix/apr_pools.c:793 >>>>>>>>> #2 0x0000560ce83e5718 in ap_push_pool (queue_info=3D0x0, >>>>>>>>> pool_to_recycle=3D0x7f4580062848) at fdqueue.c:234 >>>>>>>>> #3 0x0000560ce83e0a5a in process_lingering_close = (cs=3D0x7f4580069418, >>>>>>>>> pfd=3D0x560ce8b8dda8) at event.c:1513 >>>>>>>>> #4 0x0000560ce83e4590 in listener_thread (thd=3D0x0, = dummy=3D0x549845279ce62) >>>>>>>>> at event.c:1837 >>>>>>>>> #5 0x00007f45967610a4 in start_thread () >>>>>>>>> from /lib/x86_64-linux-gnu/libpthread.so.0 >>>>>>>>> #6 0x00007f459649662d in clone () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>>>>>> (gdb) (gdb) quit >>>>>>>>>=20 >>>>>>>>> Stefan >>>>>>>>>> Am 26.02.2017 um 19:46 schrieb Stefan Priebe - Profihost AG: >>>>>>>>>> Hi Stefan, >>>>>>>>>>=20 >>>>>>>>>> currently everything fine. No segfaults. >>>>>>>>>>=20 >>>>>>>>>> Stefan >>>>>>>>>>=20 >>>>>>>>>>> Am 25.02.2017 um 20:40 schrieb Stefan Priebe - Profihost AG: >>>>>>>>>>> Hi Stefan, >>>>>>>>>>>> Am 25.02.2017 um 13:51 schrieb Stefan Eissing: >>>>>>>>>>>> Stefan, >>>>>>>>>>>>=20 >>>>>>>>>>>> whenever you have time, please deploy = https://github.com/icing/mod_h2/releases/tag/v1.9.2 >>>>>>>>>>>>=20 >>>>>>>>>>>> I added an own allocator with mutex to the http/2 main = session. That is something of a middle-ground between placing that on = the main conn (as we had in the crash free version) and 1.9.1 behaviour. = Thanks! >>>>>>>>>>>=20 >>>>>>>>>>> done. But please keep in mind that this crash might be very = rare and >>>>>>>>>>> might even have happened with v1.9.0 if we've waited more = time. >>>>>>>>>>>=20 >>>>>>>>>>> Greets, >>>>>>>>>>> Stefan >>>>>>>>>>>=20 >>>>>>>>>>>> -Stefan >>>>>>>>>>>>=20 >>>>>>>>>>>>> Am 24.02.2017 um 19:31 schrieb Stefan Priebe - Profihost = AG : >>>>>>>>>>>>>=20 >>>>>>>>>>>>> Hi Yann, >>>>>>>>>>>>>=20 >>>>>>>>>>>>> here we go: >>>>>>>>>>>>>=20 >>>>>>>>>>>>> (gdb) p *ipsub >>>>>>>>>>>>> $1 =3D {family =3D 2, sub =3D {1367753145, 0, 0, 0}, mask = =3D {4294967295, >>>>>>>>>>>>> 4294967295, 4294967295, 4294967295}} >>>>>>>>>>>>>=20 >>>>>>>>>>>>> (gdb) p *sa >>>>>>>>>>>>> $2 =3D {pool =3D 0x10b500c4ff0b0a, hostname =3D = 0x503040203030102 >>>>>>>>>>>> Cannot access memory at address 0x503040203030102>, >>>>>>>>>>>>> servname =3D 0x17d010000040405 >>>>>>>>>>>> 0x17d010000040405>, port =3D 770, family =3D 554829073, >>>>>>>>>>>>> salen =3D 319177009, ipaddr_len =3D 570909009, = addr_str_len =3D -2127424399, >>>>>>>>>>>>> ipaddr_ptr =3D 0x24f0d15215c1b142, >>>>>>>>>>>>> next =3D 0x17160a0982726233, sa =3D {sin =3D {sin_family =3D= 6424, sin_port =3D >>>>>>>>>>>>> 9498, sin_addr =3D {s_addr =3D 690497318}, >>>>>>>>>>>>> sin_zero =3D "*456789:"}, sin6 =3D {sin6_family =3D 6424, = sin6_port =3D >>>>>>>>>>>>> 9498, sin6_flowinfo =3D 690497318, sin6_addr =3D {__in6_u = =3D { >>>>>>>>>>>>> __u6_addr8 =3D "*456789:CDEFGHIJ", __u6_addr16 =3D = {13354, 13877, >>>>>>>>>>>>> 14391, 14905, 17475, 17989, 18503, 19017}, __u6_addr32 =3D = { >>>>>>>>>>>>> 909456426, 976828471, 1178944579, 1246316615}}}, >>>>>>>>>>>>> sin6_scope_id =3D 1448432723}, sas =3D {ss_family =3D = 6424, >>>>>>>>>>>>> __ss_align =3D 4195446337656140842, >>>>>>>>>>>>> __ss_padding =3D >>>>>>>>>>>>> = "CDEFGHIJSTUVWXYZcdefghijstuvwxyz\203\204\205\206\207\210\211\212\222\223\= 224\225\226\227\230\231\232\242\243\244\245\246\247\250\251\252\262\263\26= 4\265\266\267\270\271\272\302\303\304\305\306\307\310\311\312\322\323\324\= 325\326\327\330\331\332\341\342\343\344\345\346\347\350\351\352\361\362\36= 3\364\365\366\367\370\371\372\377\304\000\037\001\000\003"}}} >>>>>>>>>>>>>=20 >>>>>>>>>>>>> (gdb) p *(struct in6_addr *)sa >>>>>>>>>>>>> $3 =3D {__in6_u =3D {__u6_addr8 =3D >>>>>>>>>>>>> = "\n\v\377\304\000\265\020\000\002\001\003\003\002\004\003\005", >>>>>>>>>>>>> __u6_addr16 =3D {2826, 50431, 46336, >>>>>>>>>>>>> 16, 258, 771, 1026, 1283}, __u6_addr32 =3D {3305048842, = 1094912, >>>>>>>>>>>>> 50528514, 84083714}}} >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> Stefan >>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Am 24.02.2017 um 14:18 schrieb Yann Ylavic: >>>>>>>>>>>>>> Hi Stefan (Priebe), >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Is IPv6 (really) involved in your network? >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Could you please show up the gdb output of the below ? >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>> On Fri, Feb 24, 2017 at 2:07 PM, Yann Ylavic = wrote: >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>> 1078 APR_DECLARE(int) apr_ipsubnet_test(apr_ipsubnet_t = *ipsub, >>>>>>>>>>>>>>> apr_sockaddr_t *sa) >>>>>>>>>>>>>>> 1079 { >>>>>>>>>>>>>>> 1080 #if APR_HAVE_IPV6 >>>>>>>>>>>>>>> 1081 /* XXX This line will segv on Win32 build with = APR_HAVE_IPV6, >>>>>>>>>>>>>>> 1082 * but without the IPV6 drivers installed. >>>>>>>>>>>>>>> 1083 */ >>>>>>>>>>>>>>> 1084 if (sa->family =3D=3D AF_INET) { >>>>>>>>>>>>>>> 1085 if (ipsub->family =3D=3D AF_INET && >>>>>>>>>>>>>>> 1086 ((sa->sa.sin.sin_addr.s_addr & = ipsub->mask[0]) =3D=3D >>>>>>>>>>>>>>> ipsub->sub[0])) { >>>>>>>>>>>>>>> 1087 return 1; >>>>>>>>>>>>>>> 1088 } >>>>>>>>>>>>>>> 1089 } >>>>>>>>>>>>>>> 1090 else if (IN6_IS_ADDR_V4MAPPED((struct in6_addr = *)sa->ipaddr_ptr)) { >>>>>>>>>>>>>>> 1091 if (ipsub->family =3D=3D AF_INET && >>>>>>>>>>>>>>> 1092 (((apr_uint32_t *)sa->ipaddr_ptr)[3] & >>>>>>>>>>>>>>> ipsub->mask[0]) =3D=3D ipsub->sub[0]) { >>>>>>>>>>>>>>> 1093 return 1; >>>>>>>>>>>>>>> 1094 } >>>>>>>>>>>>>>> 1095 } >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> (gdb) p *ipsub >>>>>>>>>>>>>> (gdb) p *sa >>>>>>>>>>>>>> (gdb) p *(struct in6_addr *)sa >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> and possibly more to come... >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Yann. >>>>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>> Stefan Eissing >>>>>>>>>>>>=20 >>>>>>>>>>>> bytes GmbH >>>>>>>>>>>> Hafenstrasse 16 >>>>>>>>>>>> 48155 M=C3=BCnster >>>>>>>>>>>> www.greenbytes.de >>>>>>>>>>>>=20 >>>>>>>>=20 >>>>>>>> Stefan Eissing >>>>>>>>=20 >>>>>>>> bytes GmbH >>>>>>>> Hafenstrasse 16 >>>>>>>> 48155 M=C3=BCnster >>>>>>>> www.greenbytes.de >>>>>>>>=20 >>>>>>=20 >>>>>> Stefan Eissing >>>>>>=20 >>>>>> bytes GmbH >>>>>> Hafenstrasse 16 >>>>>> 48155 M=C3=BCnster >>>>>> www.greenbytes.de >>>>>>=20 >>>=20 Stefan Eissing bytes GmbH Hafenstrasse 16 48155 M=C3=BCnster www.greenbytes.de