Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id A6E06200C01 for ; Thu, 19 Jan 2017 22:44:41 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id A5594160B54; Thu, 19 Jan 2017 21:44:41 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id A32F1160B3A for ; Thu, 19 Jan 2017 22:44:40 +0100 (CET) Received: (qmail 75170 invoked by uid 500); 19 Jan 2017 21:44:39 -0000 Mailing-List: contact dev-help@httpd.apache.org; run by ezmlm Precedence: bulk Reply-To: dev@httpd.apache.org list-help: list-unsubscribe: List-Post: List-Id: Delivered-To: mailing list dev@httpd.apache.org Received: (qmail 75160 invoked by uid 99); 19 Jan 2017 21:44:39 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 19 Jan 2017 21:44:39 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 3E022188877 for ; Thu, 19 Jan 2017 21:44:39 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -3.1 X-Spam-Level: X-Spam-Status: No, score=-3.1 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RP_MATCHES_RCVD=-2.999, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=greenbytes.de header.b=bESMRBFl; dkim=pass (1024-bit key) header.d=greenbytes.de header.b=m07SCLfV Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id Egz1qAvd9v6y for ; Thu, 19 Jan 2017 21:44:33 +0000 (UTC) Received: from mail.greenbytes.de (mail.greenbytes.de [5.10.171.186]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id D7E805F3BC for ; Thu, 19 Jan 2017 21:44:32 +0000 (UTC) Received: by mail.greenbytes.de (Postfix, from userid 117) id 954FA15A00E1; Thu, 19 Jan 2017 22:44:31 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=greenbytes.de; s=mail; t=1484862271; bh=maFkWd40he0XR/FV1YIoZN2mNj/t+jpFhSY/Fy9epo0=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From; b=bESMRBFlEdxExXV29hsIHTRAfAC37RFiKRuBPEIhdoCKJa1h/tnAe+EH4QoyMUCOS 0JyLbVAKkA0e1+3ifw9QNUq0g3kFrMs6VYUHNGiG4ya/OREGZa+OvJDox6lx38I1m9 Yd0nm1nv7hNSM8ioDFT11ITN5mTes8uXdrEbUpTI= Received: from [192.168.178.72] (unknown [93.211.112.192]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mail.greenbytes.de (Postfix) with ESMTPSA id 177A415A00E1; Thu, 19 Jan 2017 22:44:30 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=greenbytes.de; s=mail; t=1484862270; bh=maFkWd40he0XR/FV1YIoZN2mNj/t+jpFhSY/Fy9epo0=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From; b=m07SCLfVm0ll+f/SKO3qqnQKOJ/mc0s4PQo7sWORIg8fsdWtqt2iAN5QD6+5Buv0B aGdNMZvY3jJs99trJoHoA0Btqi0vC0/7lC7dxsn0CKcAUFYrf38nj7nCgHUw11TO/v 2Or+2HQjV4vkUCBREO3EEO9aQ5gzGXBc7pAM0nLA= Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: Re: mod_http2 and Frequent wake-ups for mpm_event From: Stefan Eissing In-Reply-To: <2c756984-dbc6-430e-9062-da99dc48632c@profihost.ag> Date: Thu, 19 Jan 2017 22:44:29 +0100 Cc: Yann Ylavic Content-Transfer-Encoding: quoted-printable Message-Id: <24436075-EF8F-44A1-A608-AB82234CD20B@greenbytes.de> References: <537782fd-3d42-9278-5750-895e83aa66fb@profihost.ag> <4b847a8c-4b2e-2a9e-a8c2-231751f4d685@profihost.ag> <1c92c16b-fc74-1ea9-780a-234d215e677d@profihost.ag> <88FB197F-71A2-45E7-89D7-01AB52155D87@greenbytes.de> <2e62a382-a26c-1650-883b-3278b6a43ccc@profihost.ag> <074df1d4-93a6-72dd-e321-7e12f2c6432b@profihost.ag> <44421233-0632-4E17-96A2-F6E7936405B5@greenbytes.de> <90ce5069-9a64-6306-bb8c-361e3dd019c8@profihost.ag> <84c88399-2914-7911-2828-173cbca3a090@profihost.ag> <9249A8F9-0446-40ED-B5B6-5726A68AE88E@greenbytes.de> <1b306b5b-0124-162d-d786-550eccd75ef8@profihost.ag> <12F4ADE8-BFA6-4C3A-9F58-00E367B212F7@greenbytes.de> <2c756984-dbc6-430e-9062-da99dc48632c@profihost.ag> To: dev@httpd.apache.org X-Mailer: Apple Mail (2.3259) archived-at: Thu, 19 Jan 2017 21:44:41 -0000 Stefan, this seems to be a tough bone to chew. Therefore we need to go deeper: - can you compile the module so that we see line numbers in the trace? - which apr version are you using? - can you reproduce this at will? How? Which client? Just a GET or = something more sophisticated? Thanks for the help! Cheers, Stefan > Am 19.01.2017 um 22:01 schrieb Stefan Priebe - Profihost AG = : >=20 > new segfault with both patches on top of v1.8.8: > Using host libthread_db library = "/lib/x86_64-linux-gnu/libthread_db.so.1". > Core was generated by `/usr/local/apache2/bin/httpd -k start'. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 0x00007fbefce0d014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #0 0x00007fbefce0d014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #1 0x00007fbefd2a0036 in ?? () from = /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #2 0x00007fbefd2a046f in apr_hash_set () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #3 0x000000000052a26b in h2_ihash_remove () > #4 0x0000000000506b24 in purge_stream () > #5 0x000000000052a1c2 in ihash_iter () > #6 0x00007fbefd2a08a6 in apr_hash_do () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #7 0x000000000052a200 in h2_ihash_iter () > #8 0x0000000000506b65 in purge_streams () > #9 0x00000000005082dd in h2_mplx_release_and_join () > #10 0x00000000005158f9 in h2_session_cleanup () > #11 0x00000000005164a4 in session_pool_cleanup () > #12 0x00007fbefd2a9976 in apr_pool_destroy () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #13 0x00007fbefd2a9c55 in apr_pool_clear () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #14 0x0000000000566acd in ap_push_pool () > #15 0x000000000056008a in process_lingering_close () > #16 0x0000000000560ced in listener_thread () > #17 0x00007fbefd0780a4 in start_thread () from > /lib/x86_64-linux-gnu/libpthread.so.0 > #18 0x00007fbefcdad62d in clone () from = /lib/x86_64-linux-gnu/libc.so.6 >=20 > Stefan >=20 > Am 19.01.2017 um 21:48 schrieb Stefan Eissing: >> On top please. There is only one way: forward! >>=20 >>> Am 19.01.2017 um 21:47 schrieb Stefan Priebe - Profihost AG = : >>>=20 >>>=20 >>> Am 19.01.2017 um 21:39 schrieb Stefan Eissing: >>>> Thanks, Stefan. Can you given the attached Patch a try?=20 >>>=20 >>> sure. On top of the last one? Or should i drop it? >>>=20 >>>=20 >>> Stefan >>>=20 >>>>> Am 19.01.2017 um 19:33 schrieb Stefan Priebe = : >>>>>=20 >>>>> Here some more segfaults from 2.4.25 no mpm patch but http2 = v1.8.8: >>>>>=20 >>>>> ################################################################# >>>>>=20 >>>>> Using host libthread_db library = "/lib/x86_64-linux-gnu/libthread_db.so.1". >>>>> Core was generated by `/usr/local/apache2/bin/httpd -k start'. >>>>> Program terminated with signal SIGSEGV, Segmentation fault. >>>>> #0 0x00007f61f673b014 in ?? () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>> #0 0x00007f61f673b014 in ?? () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>> #1 0x00007f61f6bce036 in ?? () from = /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>>>> #2 0x00007f61f6bce46f in apr_hash_set () from = /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>>>> #3 0x000000000052a26d in h2_ihash_remove () >>>>> #4 0x0000000000506b24 in purge_stream () >>>>> #5 0x000000000052a1c4 in ihash_iter () >>>>> #6 0x00007f61f6bce8a6 in apr_hash_do () from = /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>>>> #7 0x000000000052a202 in h2_ihash_iter () >>>>> #8 0x0000000000506b65 in purge_streams () >>>>> #9 0x00000000005082df in h2_mplx_release_and_join () >>>>> #10 0x00000000005158fb in h2_session_cleanup () >>>>> #11 0x00000000005164a6 in session_pool_cleanup () >>>>> #12 0x00007f61f6bd7976 in apr_pool_destroy () from = /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>>>> #13 0x00007f61f6bd7c55 in apr_pool_clear () from = /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>>>> #14 0x0000000000566acf in ap_push_pool () >>>>> #15 0x000000000056008c in process_lingering_close () >>>>> #16 0x0000000000560cef in listener_thread () >>>>> #17 0x00007f61f69a60a4 in start_thread () from = /lib/x86_64-linux-gnu/libpthread.so.0 >>>>> #18 0x00007f61f66db62d in clone () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>>=20 >>>>> ################################################################# >>>>>=20 >>>>> Using host libthread_db library = "/lib/x86_64-linux-gnu/libthread_db.so.1". >>>>> Core was generated by `/usr/local/apache2/bin/httpd -k start'. >>>>> Program terminated with signal SIGSEGV, Segmentation fault. >>>>> #0 0x00007f61f673b014 in ?? () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>> #0 0x00007f61f673b014 in ?? () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>> #1 0x00007f61f6bce036 in ?? () from = /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>>>> #2 0x00007f61f6bce46f in apr_hash_set () from = /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>>>> #3 0x000000000052a26d in h2_ihash_remove () >>>>> #4 0x0000000000506b24 in purge_stream () >>>>> #5 0x000000000052a1c4 in ihash_iter () >>>>> #6 0x00007f61f6bce8a6 in apr_hash_do () from = /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>>>> #7 0x000000000052a202 in h2_ihash_iter () >>>>> #8 0x0000000000506b65 in purge_streams () >>>>> #9 0x00000000005082df in h2_mplx_release_and_join () >>>>> #10 0x00000000005158fb in h2_session_cleanup () >>>>> #11 0x00000000005164a6 in session_pool_cleanup () >>>>> #12 0x00007f61f6bd7976 in apr_pool_destroy () from = /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>>>> #13 0x00007f61f6bd7c55 in apr_pool_clear () from = /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>>>> #14 0x0000000000566acf in ap_push_pool () >>>>> #15 0x000000000056008c in process_lingering_close () >>>>> #16 0x0000000000560cef in listener_thread () >>>>> #17 0x00007f61f69a60a4 in start_thread () from = /lib/x86_64-linux-gnu/libpthread.so.0 >>>>> #18 0x00007f61f66db62d in clone () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>>=20 >>>>> ################################################################# >>>>>=20 >>>>> Program terminated with signal SIGSEGV, Segmentation fault. >>>>> #0 0x00007f204f922d63 in apr_pool_cleanup_kill () from = /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>>>> (gdb) bt >>>>> #0 0x00007f204f922d63 in apr_pool_cleanup_kill () from = /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>>>> #1 0x00007f204f922e21 in apr_pool_cleanup_run () from = /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>>>> #2 0x000000000055ffe9 in process_lingering_close () >>>>> #3 0x0000000000560cef in listener_thread () >>>>> #4 0x00007f204f6f00a4 in start_thread () from = /lib/x86_64-linux-gnu/libpthread.so.0 >>>>> #5 0x00007f204f42562d in clone () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>>=20 >>>>> full bt: >>>>> http://pastebin.com/raw/bP1vaYjw >>>>>=20 >>>>> ################################################################# >>>>>=20 >>>>> Greets, >>>>> Stefan >>>>>=20 >>>>> Am 19.01.2017 um 16:47 schrieb Stefan Priebe - Profihost AG: >>>>>> arg sorry my fault. >>>>>>=20 >>>>>> Here is a complete trace: >>>>>> Program terminated with signal SIGSEGV, Segmentation fault. >>>>>> #0 0x00007fc1c23e0f23 in apr_brigade_length () from >>>>>> /usr/lib/x86_64-linux-gnu/libaprutil-1.so.0 >>>>>> (gdb) bt >>>>>> #0 0x00007fc1c23e0f23 in apr_brigade_length () from >>>>>> /usr/lib/x86_64-linux-gnu/libaprutil-1.so.0 >>>>>> #1 0x000000000052a8f1 in h2_util_bb_avail () >>>>>> #2 0x0000000000521eaa in h2_stream_out_prepare () >>>>>> #3 0x000000000051a55a in on_stream_resume () >>>>>> #4 0x000000000050bdff in h2_mplx_dispatch_master_events () >>>>>> #5 0x000000000051dc32 in h2_session_process () >>>>>> #6 0x0000000000500115 in h2_conn_run () >>>>>> #7 0x0000000000504b51 in h2_h2_process_conn () >>>>>> #8 0x000000000047cb19 in ap_run_process_connection () >>>>>> #9 0x000000000055e755 in process_socket () >>>>>> #10 0x0000000000560f5c in worker_thread () >>>>>> #11 0x00007fc1c1f8d0a4 in start_thread () from >>>>>> /lib/x86_64-linux-gnu/libpthread.so.0 >>>>>> #12 0x00007fc1c1cc262d in clone () from = /lib/x86_64-linux-gnu/libc.so.6 >>>>>>=20 >>>>>> Stefan >>>>>>=20 >>>>>> Am 19.01.2017 um 16:45 schrieb Stefan Priebe - Profihost AG: >>>>>>>=20 >>>>>>> Am 19.01.2017 um 16:34 schrieb Stefan Eissing: >>>>>>>> Yann might already have asked this: any chance to compile with = symbols and get a more readable stacktrace? >>>>>>>=20 >>>>>>> Yes just tell me how ;-) i'm using dpkg-buildpackage and = dh_strip. I've >>>>>>> no idea why not all symbols are available. >>>>>>>=20 >>>>>>> Do i need to pass a specific option to configure >>>>>>>=20 >>>>>>> Stefan >>>>>>>=20 >>>>>>>>> Am 19.01.2017 um 16:30 schrieb Stefan Priebe - Profihost AG = : >>>>>>>>>=20 >>>>>>>>> With stock 2.4.25 + patch i'm getting this one again: >>>>>>>>> (gdb) bt >>>>>>>>> #0 0x0000000000521dcd in h2_stream_out_prepare () >>>>>>>>> #1 0x00007fc1a2feca80 in ?? () >>>>>>>>> #2 0x00007fc1a2feca8c in ?? () >>>>>>>>> #3 0x00007fc1a2feca90 in ?? () >>>>>>>>> #4 0x00007fc1a057c0a0 in ?? () >>>>>>>>> #5 0x00007fc1a057cdd8 in ?? () >>>>>>>>> #6 0x00007fc1a2fecac0 in ?? () >>>>>>>>> #7 0x0000000000000000 in ?? () >>>>>>>>>=20 >>>>>>>>> Stefan >>>>>>>>>=20 >>>>>>>>> Am 19.01.2017 um 16:28 schrieb Stefan Priebe - Profihost AG: >>>>>>>>>> I'm now testing stock 2.4.25 + patch. >>>>>>>>>>=20 >>>>>>>>>> May this configure option have an influence? >>>>>>>>>> --enable-nonportable-atomics=3Dyes >>>>>>>>>>=20 >>>>>>>>>> Greets, >>>>>>>>>> Stefan >>>>>>>>>>=20 >>>>>>>>>> Am 19.01.2017 um 15:35 schrieb Yann Ylavic: >>>>>>>>>>> Hi, >>>>>>>>>>>=20 >>>>>>>>>>> On Thu, Jan 19, 2017 at 3:00 PM, Stefan Priebe - Profihost = AG >>>>>>>>>>> wrote: >>>>>>>>>>>>=20 >>>>>>>>>>>> @Yann: >>>>>>>>>>>> should i use V7 or V6? >>>>>>>>>>>=20 >>>>>>>>>>> I'd prefer you'd use none (such that we can verify the patch = with >>>>>>>>>>> stock 2.4.25, modulo mod_http2), but if it's easier for you = to >>>>>>>>>>> reproduce with an event's patch, please use the v6 (and if = it fails >>>>>>>>>>> then v7, and if it fails then no patch, really). >>>>>>>>>>>=20 >>>>>>>>=20 >>>>>>>> Stefan Eissing >>>>>>>>=20 >>>>>>>> bytes GmbH >>>>>>>> Hafenstrasse 16 >>>>>>>> 48155 M=C3=BCnster >>>>>>>> www.greenbytes.de >>>>>>>>=20 >>>>=20 >>>> Stefan Eissing >>>>=20 >>>> bytes GmbH >>>> Hafenstrasse 16 >>>> 48155 M=C3=BCnster >>>> www.greenbytes.de >>>>=20 >>=20 >> Stefan Eissing >>=20 >> bytes GmbH >> Hafenstrasse 16 >> 48155 M=C3=BCnster >> www.greenbytes.de >>=20 Stefan Eissing bytes GmbH Hafenstrasse 16 48155 M=C3=BCnster www.greenbytes.de