From dev-return-195121-archive-asf-public=cust-asf.ponee.io@tomcat.apache.org Wed Nov 28 16:52:36 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 2E9D5180658 for ; Wed, 28 Nov 2018 16:52:36 +0100 (CET) Received: (qmail 47260 invoked by uid 500); 28 Nov 2018 15:52:35 -0000 Mailing-List: contact dev-help@tomcat.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Tomcat Developers List" Delivered-To: mailing list dev@tomcat.apache.org Received: (qmail 47250 invoked by uid 99); 28 Nov 2018 15:52:34 -0000 Received: from mail-relay.apache.org (HELO mailrelay1-lw-us.apache.org) (207.244.88.152) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 28 Nov 2018 15:52:34 +0000 Received: from [192.168.23.12] (host86-152-79-27.range86-152.btcentralplus.com [86.152.79.27]) by mailrelay1-lw-us.apache.org (ASF Mail Server at mailrelay1-lw-us.apache.org) with ESMTPSA id 11ED9F69 for ; Wed, 28 Nov 2018 15:52:33 +0000 (UTC) Subject: Re: svn commit: r1847646 - /tomcat/trunk/java/org/apache/tomcat/util/net/Nio2Endpoint.java To: dev@tomcat.apache.org References: <20181128150007.00B513A0102@svn01-us-west.apache.org> <0a9d4277-ed6b-cd5e-b5de-5891d84868cd@apache.org> From: Mark Thomas Openpgp: preference=signencrypt Autocrypt: addr=markt@apache.org; prefer-encrypt=mutual; keydata= xsFNBEq0DukBEAD4jovHOPJDxoD+JnO1Go2kiwpgRULasGlrVKuSUdP6wzcaqWmXpqtOJKKw W2MQFQLmg7nQ9RjJwy3QCbKNDJQA/bwbQT1F7WzTCz2S6vxC4zxKck4t6RZBq2dJsYKF0CEh 6ZfY4dmKvhq+3istSoFRdHYoOPGWZpuRDqfZPdGm/m335/6KGH59oysn1NE7a2a+kZzjBSEg v23+l4Z1Rg7+fpz1JcdHSdC2Z+ZRxML25eVatRVz4yvDOZItqDURP24zWOodxgboldV6Y88C 3v/7KRR+1vklzkuA2FqF8Q4r/2f0su7MUVviQcy29y/RlLSDTTYoVlCZ1ni14qFU7Hpw43KJ tgXmcUwq31T1+SlXdYjNJ1aFkUi8BjCHDcSgE/IReKUanjHzm4XSymKDTeqqzidi4k6PDD4j yHb8k8vxi6qT6Udnlcfo5NBkkUT1TauhEy8ktHhbl9k60BvvMBP9l6cURiJg1WS77egI4P/8 2oPbzzFiGFqXyJKULVgxtdQ3JikCpodp3f1fh6PlYZwkW4xCJLJucJ5MiQp07HAkMVW5w+k8 Xvuk4i5quh3N+2kzKHOOiQCDmN0sz0XjOE+7XBvM1lvz3+UarLfgSVmW8aheLd7eaIl5ItBk 8844ZJ60LrQ+JiIqvqJemxyIM6epoZvY5a3ZshZpcLilC5hW8QARAQABzSJNYXJrIEUgRCBU aG9tYXMgPG1hcmt0QGFwYWNoZS5vcmc+wsF3BBMBCgAhBQJKtA7pAhsDBQsJCAcDBRUKCQgL BRYCAwEAAh4BAheAAAoJEBDAHFovYFnn2YgQAKN6FLG/I1Ij3PUlC/XNlhasQxPeE3w2Ovtt weOQPYkblJ9nHtGH5pNqG2/qoGShlpI04jJy9GxWKOo7NV4v7M0mbVlCXVgjdlvMFWdL7lno cggwJAFejQcYlVtxyhu4m50LBvBunEhxCbQcKnnWmkB7Ocm0Ictaqjc9rCc1F/aNhVMUpJ0z G1kyTp9hxvN6TbCQlacMx5ocTWzL0zn6QZhbUfrYwfxYJmSnkVYZOYzXIXIsLN5sJ9Q4P8tj Y4qWgd+bQvOqPWrkzL9LVRnGOrSYIsoM5zWdoj1g1glMzK/ZqJdRqqqBhe6FYTbXipz8oX8i mCebcaxZnfLhGiqqX+yDa3YUwDiqom+sZOc0iXGvKkqltPLpNeF0MVT7aZjalsQ/v2Ysb24R Ql9FfjfWmvT8ZPWz8Kore1AI4UcIIgFVtM+zuLlL9CIsGjg+gHDE2dhZDY0qfizlHL9CoAWU DM3pIfxM2V4BRn1xO+j/mModhjmYLZvnFVz4KGkNO7wRkofAANIWYo3WI5x83BGDH371t3NR rrpSSFP0XpQX6/Leaj2j6U6puABL2qBxhscsO6chc3u4/+019ff+peZVsc9ttcTQXsKIujmM b8p2sk5usmv6PKVX3oW/RAxpbVHU5kZ5px1Hq7mMQdZfLs5ff4YymXBH02z4/RmSzPam0Xb5 zsFNBEq0DukBEADCNEkws5YroBmbu8789Xf006gTl5LzD/Hdt3sAp9iCfPgucO+l7U+xbo1X HTMJQwEVfS+Rx3RbaLYRG+hU7FuJLQB/5NaCDNRuqw5KHyQtJUH+zo84IqqfMzG8aOSdHg1y r2xKH4QTmgQONBu/W0xEZmZro6TjYNwkk2pwXK2yuImZPUOy+mK1qF8Wm3hTtkPE+FFSNFIa eHDoTGmx/0Riu/K7dNJTrC0TlRpn2K6d60zB53YYTc+0DYSDyB0FupXiAx/+XEGn3Q7eNi2B V6w50v5r51QP8zptiFflMfFKNAfV8xS5MteQd98YS5qqd/LPo3gS5HFPQaSL0k3RTClv7fQN HcZFqmv0OWpix6zm2npYxhqsTDGeSa52/uXehVXF5JubYFifMSLpbGVZqdrmG5hr2cycxsjF iY0zJOaRitmN/JWbOGLiwrcN4ukKNyFntFG5jPaFnJdx9rHfyJNeF9cgv9JlZeFxJ6WqIAhl KOuH3K8/py0SPE6ZOFfRo0YUxvh25K/siOcPLm613aOxyY7YfQ8ME2vgn7I0mAtg9am+YFDa bGqj839odwZdzZv2T2mUHnybFTJFBuMWGWKYstYDS6eZEmhupbPvUKkDug/mO+gdo+pSKF9Y S6DM5RtCdTNJq4NZY50ypBb5RSj+INHPocIp2V/DDTbzySsu6wARAQABwsFfBBgBCgAJBQJK tA7pAhsMAAoJEBDAHFovYFnnLe0P/i34oK5cE2LlqUEITEcTO94x1EX0UmtKokRfQ3AYWK8X eFD8cmSty72hMkL+1c0V//4Qc53SUyLIWXk8FKWF7hdL3zyuBqlRb55721CYC35GA/jR90p0 k1vr701gaat2cNTOVC0/6H9cE5yYXT+zMr9TSiKCDwONhhSbmAJZc6X0fgsmCD7I5xUI5Vri hN/Wx0CZBtrXGUyE4hgFaYSGptZmkY5Ln1e+nI185Bda7bpLwcAIGrI9nYtVXgf71ybGKdPP tFfXIoPXuctn99M7NnWBhNuGDms2YWkOC7eeWBTxKkZDWR3vRmRy52B6GxR7USk/KXs7yqGP kfT/c4CZFfOurZUXXuC3PvOme0DQmqwExtJormoG4Fy6suEFPrfhYMigTy7kSbVTCOBMjQLH +U/FFNshvg9+M/ZvaKT+0lpRvBSuG5ngsC0bO0xWsXhb6qfH2h53g4VcwFvCBL5IfqgAeUbC nGGHNcGWpmwdeb7D7ahrNZSHEUUYR7lTbjkYS01/QDOcEwNZOqDRIJUQOOUq35721VeROkdh ZmMZtFlsQeQJsWoqGrQo/kEYicVlMVOgjmOOzOa5fRb/IqlGlBn4a4me3hWthLLtMy+OOEim 6ENjntVTBQiTP/YqrxWDbCkaD7b2e9wY5N3JlRxMIQHfcHaND3PRdQSn7oHYXmJl Message-ID: Date: Wed, 28 Nov 2018 15:52:32 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 8bit On 28/11/2018 15:30, Rémy Maucherat wrote: > On Wed, Nov 28, 2018 at 4:20 PM Mark Thomas wrote: > >> On 28/11/2018 15:00, markt@apache.org wrote: >>> Author: markt >>> Date: Wed Nov 28 15:00:06 2018 >>> New Revision: 1847646 >>> >>> URL: http://svn.apache.org/viewvc?rev=1847646&view=rev >>> Log: >>> Fix possible cause of intermittent TestCoyoteOutputStream failures. >> >> I thought this would be worthy of a longer explanation than seemed >> appropriate for a commit message. >> > > I feel bad for not thinking about it since it does sound quite logical. I wouldn't feel too bad. I've lost track of the number of logical explanations I have had for problems that I have had to throw away once I had more data. To be honest, I'm expecting this to carry on failing intermittently and that the root cause - when we find it - will turn out to be my refactoring. > BTW, the testsuite failed but it wasn't *that*. What a coincidence ! Indeed. I saw that and started to mentally draft a "Ignore my last" email while I scrolled to the end to see which test had failed. Mark > > Rémy > > >> >> I have tried to recreate the issue locally without success. I was able >> to recreate it occasionally running the tests on silvanus.a.o (the CI >> machine that runs all our buildbot jobs). >> >> I captured a network trace that confirmed that this was a server side >> bug. What I saw was a corrupted response. The headers and first chunk >> were correct but rather than the 5 bytes of the end chunk I saw the >> other 8187 (8192-5) bytes of the buffer. It was clear the buffer was >> configured for write when it was being read. >> >> I then tried to figure out how this could happen with a view to >> reproducing the issue. >> >> There were a lot of dead ends during which I noticed that the write >> pattern varied when I added additional debug statements. I discovered >> that, depending on timing, the NIO2 endpoint would sometimes use a >> gathering write when performing a non-blocking flush. >> >> There is a non-blocking flush just before the switch back to blocking >> I/O (after the dispatch to end the async component) and it looked to be >> possible that the gathering write could still be in progress when the >> following blocking write was performed. That in turn meant that one of >> the buffers used by the gathering write could be modified during the >> following blocking write. >> >> However, my current understanding of the code is that the gathering >> write will have written all the data from the buffer that is used by the >> following blocking write before that blocking write occurs. So I may >> have missed the root cause completely. It depends a lot on the internal >> workings of the AsynchronousSocketChannel. >> >> On balance, I decided to commit this fix as there does appear to be a >> bug here. Hopefully, it is the root cause of the intermittent >> TestCoyoteOutputStream failures. If it is, great. If not, I'll keep >> looking. >> >> Mark >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org >> For additional commands, e-mail: dev-help@tomcat.apache.org >> >> > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For additional commands, e-mail: dev-help@tomcat.apache.org