From dev-return-193706-archive-asf-public=cust-asf.ponee.io@tomcat.apache.org Wed Oct 3 11:29:56 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 7A81118065B for ; Wed, 3 Oct 2018 11:29:55 +0200 (CEST) Received: (qmail 78457 invoked by uid 500); 3 Oct 2018 09:29:54 -0000 Mailing-List: contact dev-help@tomcat.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Tomcat Developers List" Delivered-To: mailing list dev@tomcat.apache.org Received: (qmail 78447 invoked by uid 99); 3 Oct 2018 09:29:54 -0000 Received: from mail-relay.apache.org (HELO mailrelay2-lw-us.apache.org) (207.244.88.137) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Oct 2018 09:29:54 +0000 Received: from [192.168.23.12] (host86-152-79-27.range86-152.btcentralplus.com [86.152.79.27]) by mailrelay2-lw-us.apache.org (ASF Mail Server at mailrelay2-lw-us.apache.org) with ESMTPSA id 511E8469 for ; Wed, 3 Oct 2018 09:29:53 +0000 (UTC) Subject: Re: SSL Unit Tests Failing To: Tomcat Developers List References: <61cdb603-8d44-28f8-302a-daff2b186dd4@apache.org> <8e73c4f4-a5d0-2ebe-ca55-3b56dc17b9f3@apache.org> From: Mark Thomas Openpgp: preference=signencrypt Autocrypt: addr=markt@apache.org; prefer-encrypt=mutual; keydata= xsFNBEq0DukBEAD4jovHOPJDxoD+JnO1Go2kiwpgRULasGlrVKuSUdP6wzcaqWmXpqtOJKKw W2MQFQLmg7nQ9RjJwy3QCbKNDJQA/bwbQT1F7WzTCz2S6vxC4zxKck4t6RZBq2dJsYKF0CEh 6ZfY4dmKvhq+3istSoFRdHYoOPGWZpuRDqfZPdGm/m335/6KGH59oysn1NE7a2a+kZzjBSEg v23+l4Z1Rg7+fpz1JcdHSdC2Z+ZRxML25eVatRVz4yvDOZItqDURP24zWOodxgboldV6Y88C 3v/7KRR+1vklzkuA2FqF8Q4r/2f0su7MUVviQcy29y/RlLSDTTYoVlCZ1ni14qFU7Hpw43KJ tgXmcUwq31T1+SlXdYjNJ1aFkUi8BjCHDcSgE/IReKUanjHzm4XSymKDTeqqzidi4k6PDD4j yHb8k8vxi6qT6Udnlcfo5NBkkUT1TauhEy8ktHhbl9k60BvvMBP9l6cURiJg1WS77egI4P/8 2oPbzzFiGFqXyJKULVgxtdQ3JikCpodp3f1fh6PlYZwkW4xCJLJucJ5MiQp07HAkMVW5w+k8 Xvuk4i5quh3N+2kzKHOOiQCDmN0sz0XjOE+7XBvM1lvz3+UarLfgSVmW8aheLd7eaIl5ItBk 8844ZJ60LrQ+JiIqvqJemxyIM6epoZvY5a3ZshZpcLilC5hW8QARAQABzSJNYXJrIEUgRCBU aG9tYXMgPG1hcmt0QGFwYWNoZS5vcmc+wsF3BBMBCgAhBQJKtA7pAhsDBQsJCAcDBRUKCQgL BRYCAwEAAh4BAheAAAoJEBDAHFovYFnn2YgQAKN6FLG/I1Ij3PUlC/XNlhasQxPeE3w2Ovtt weOQPYkblJ9nHtGH5pNqG2/qoGShlpI04jJy9GxWKOo7NV4v7M0mbVlCXVgjdlvMFWdL7lno cggwJAFejQcYlVtxyhu4m50LBvBunEhxCbQcKnnWmkB7Ocm0Ictaqjc9rCc1F/aNhVMUpJ0z G1kyTp9hxvN6TbCQlacMx5ocTWzL0zn6QZhbUfrYwfxYJmSnkVYZOYzXIXIsLN5sJ9Q4P8tj Y4qWgd+bQvOqPWrkzL9LVRnGOrSYIsoM5zWdoj1g1glMzK/ZqJdRqqqBhe6FYTbXipz8oX8i mCebcaxZnfLhGiqqX+yDa3YUwDiqom+sZOc0iXGvKkqltPLpNeF0MVT7aZjalsQ/v2Ysb24R Ql9FfjfWmvT8ZPWz8Kore1AI4UcIIgFVtM+zuLlL9CIsGjg+gHDE2dhZDY0qfizlHL9CoAWU DM3pIfxM2V4BRn1xO+j/mModhjmYLZvnFVz4KGkNO7wRkofAANIWYo3WI5x83BGDH371t3NR rrpSSFP0XpQX6/Leaj2j6U6puABL2qBxhscsO6chc3u4/+019ff+peZVsc9ttcTQXsKIujmM b8p2sk5usmv6PKVX3oW/RAxpbVHU5kZ5px1Hq7mMQdZfLs5ff4YymXBH02z4/RmSzPam0Xb5 zsFNBEq0DukBEADCNEkws5YroBmbu8789Xf006gTl5LzD/Hdt3sAp9iCfPgucO+l7U+xbo1X HTMJQwEVfS+Rx3RbaLYRG+hU7FuJLQB/5NaCDNRuqw5KHyQtJUH+zo84IqqfMzG8aOSdHg1y r2xKH4QTmgQONBu/W0xEZmZro6TjYNwkk2pwXK2yuImZPUOy+mK1qF8Wm3hTtkPE+FFSNFIa eHDoTGmx/0Riu/K7dNJTrC0TlRpn2K6d60zB53YYTc+0DYSDyB0FupXiAx/+XEGn3Q7eNi2B V6w50v5r51QP8zptiFflMfFKNAfV8xS5MteQd98YS5qqd/LPo3gS5HFPQaSL0k3RTClv7fQN HcZFqmv0OWpix6zm2npYxhqsTDGeSa52/uXehVXF5JubYFifMSLpbGVZqdrmG5hr2cycxsjF iY0zJOaRitmN/JWbOGLiwrcN4ukKNyFntFG5jPaFnJdx9rHfyJNeF9cgv9JlZeFxJ6WqIAhl KOuH3K8/py0SPE6ZOFfRo0YUxvh25K/siOcPLm613aOxyY7YfQ8ME2vgn7I0mAtg9am+YFDa bGqj839odwZdzZv2T2mUHnybFTJFBuMWGWKYstYDS6eZEmhupbPvUKkDug/mO+gdo+pSKF9Y S6DM5RtCdTNJq4NZY50ypBb5RSj+INHPocIp2V/DDTbzySsu6wARAQABwsFfBBgBCgAJBQJK tA7pAhsMAAoJEBDAHFovYFnnLe0P/i34oK5cE2LlqUEITEcTO94x1EX0UmtKokRfQ3AYWK8X eFD8cmSty72hMkL+1c0V//4Qc53SUyLIWXk8FKWF7hdL3zyuBqlRb55721CYC35GA/jR90p0 k1vr701gaat2cNTOVC0/6H9cE5yYXT+zMr9TSiKCDwONhhSbmAJZc6X0fgsmCD7I5xUI5Vri hN/Wx0CZBtrXGUyE4hgFaYSGptZmkY5Ln1e+nI185Bda7bpLwcAIGrI9nYtVXgf71ybGKdPP tFfXIoPXuctn99M7NnWBhNuGDms2YWkOC7eeWBTxKkZDWR3vRmRy52B6GxR7USk/KXs7yqGP kfT/c4CZFfOurZUXXuC3PvOme0DQmqwExtJormoG4Fy6suEFPrfhYMigTy7kSbVTCOBMjQLH +U/FFNshvg9+M/ZvaKT+0lpRvBSuG5ngsC0bO0xWsXhb6qfH2h53g4VcwFvCBL5IfqgAeUbC nGGHNcGWpmwdeb7D7ahrNZSHEUUYR7lTbjkYS01/QDOcEwNZOqDRIJUQOOUq35721VeROkdh ZmMZtFlsQeQJsWoqGrQo/kEYicVlMVOgjmOOzOa5fRb/IqlGlBn4a4me3hWthLLtMy+OOEim 6ENjntVTBQiTP/YqrxWDbCkaD7b2e9wY5N3JlRxMIQHfcHaND3PRdQSn7oHYXmJl Message-ID: Date: Wed, 3 Oct 2018 10:29:51 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <8e73c4f4-a5d0-2ebe-ca55-3b56dc17b9f3@apache.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 8bit On 02/10/18 20:40, Igal Sapir wrote: > Mark / Chris, > > On 10/2/2018 6:36 AM, Mark Thomas wrote: >> On 02/10/18 06:58, Igal Sapir wrote: >>> When trying to run the unit test cases with `ant clean test` on the >>> current >>> trunk [1] I am getting two (per connector) failures: >>> >>>      org.apache.tomcat.util.net.openssl.ciphers.TestCipher FAILED [2] >>> >>> org.apache.tomcat.util.net.openssl.ciphers.TestOpenSSLCipherConfigurationParser >>> >>> FAILED [3] >>> >>>      Server version: Apache Tomcat/9.0.13-dev >>>      Server built:   Oct 2 2018 05:24:55 UTC >>>      Server number:  9.0.13.0 >>>      OS Name:        Linux >>>      OS Version:     4.18.9-200.fc28.x86_64 >>>      Architecture:   amd64 >>>      JVM Version:    1.8.0_181-b13 >>>      JVM Vendor:     Oracle Corporation >>> >>> Am I missing something?  Other than the obvious "missing ciphers", >>> that is. >> These tests are all particularly sensitive to the versions of OpenSSL, >> Java and the implementation of Java used. >> >> Generally, those tests are there to ensure that the code that translates >> between JSSE cipher definitions and OpenSSL definitions is correct. >> >> If you see a failure it may indicate that: >> >> - the test has a bug >> >> - you are running with an older version of OpenSSL that behaves >>    differently from the latest version (we try and keep pace with the >>    latest) >> >> - OpenSSL has changed behaviour and we need to update our translation >>    code to align with it (unusual) >> >> - OpenSSL has changed behaviour and we need to update our tests to align >>    with it (most frequent). > > Thank you both for the detailed explanation.  I suspected that I should > had added the OpenSSL version to the OP.  On that Fedora machine I have > OpenSSL 1.1.0i-fips 14 Aug 2018 > > I tried the same tests on a Windows 10 machine.  Below are some > discrepancies/peculiarities that I've noticed (I'd be happy to improve > the test cases if possible): I noticed some errors on Gump overnight so this morning I have build OpenSSL 1.0.2, 1.1.0, 1.1.1 and master locally and tested them against 8.5.x and 9.0.x. I found a couple of bugs: - The ARIA ciphers were not handled correctly so testing against OpenSSL 1.1.0 was always going to fail. This has been fixed. - 8.5.x was missing some code that ensured the OpenSSL libraries as well as the binary was on the path. This meant 8.5.x tests were either going to fail or use a locally installed OpenSSL version. This has also been fixed. > On the Linux box I have OpenSSL installed and on the PATH.  On Windows I > used version OpenSSL 1.1.1  11 Sep 2018 and specified it via the > `test.openssl.path` property.  I checked the value of > `test.openssl.exists` and it showed the expected `true`.  Both Windows > and Fedora generated an output file for > test/org/apache/tomcat/util/net/openssl/TestOpenSSLConf.java [1]. Both, > however, reported "Found OpenSSL version 0x0" which I find strange? That does seem odd. I suspect either the wrong OpenSSL version or no OpenSSL version was found. > On Windows, only the output [2] for the file mentioned above is in the > output/build/logs, while on Fedora I also have output from the 3 Test > files from test/org/apache/tomcat/util/net/openssl/ciphers/. Does that > mean that these tests were not run on Windows? That seems to be a reasonable conclusion. > I wanted to check the Gump output to compare with my local results. I > found this URL, which I'm not sure if it is the right one or not - > http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk/index.html - as > it says "Project build output found here..." but without any links or > any other information. That is the build. The full output is linked just below that line but you probably want the tests which are run as a separate build for each connector. http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk-test-nio/index.html http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk-test-nio2/index.html http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk-test-apr/index.html If you scroll down for any of those pages, you'll find the individual test files for the latest run. > I would like at the very least to add the output of `openssl version` to > the Ant output, perhaps at the `test.openssl.exists` target.  If there > are no objections I will add that. +1 Mark --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For additional commands, e-mail: dev-help@tomcat.apache.org