tomcat-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Thomas <>
Subject Re: SSL Unit Tests Failing
Date Wed, 03 Oct 2018 09:29:51 GMT
On 02/10/18 20:40, Igal Sapir wrote:
> Mark / Chris,
> On 10/2/2018 6:36 AM, Mark Thomas wrote:
>> On 02/10/18 06:58, Igal Sapir wrote:
>>> When trying to run the unit test cases with `ant clean test` on the
>>> current
>>> trunk [1] I am getting two (per connector) failures:
>>> FAILED [2]
>>> FAILED [3]
>>>      Server version: Apache Tomcat/9.0.13-dev
>>>      Server built:   Oct 2 2018 05:24:55 UTC
>>>      Server number:
>>>      OS Name:        Linux
>>>      OS Version:     4.18.9-200.fc28.x86_64
>>>      Architecture:   amd64
>>>      JVM Version:    1.8.0_181-b13
>>>      JVM Vendor:     Oracle Corporation
>>> Am I missing something?  Other than the obvious "missing ciphers",
>>> that is.
>> These tests are all particularly sensitive to the versions of OpenSSL,
>> Java and the implementation of Java used.
>> Generally, those tests are there to ensure that the code that translates
>> between JSSE cipher definitions and OpenSSL definitions is correct.
>> If you see a failure it may indicate that:
>> - the test has a bug
>> - you are running with an older version of OpenSSL that behaves
>>    differently from the latest version (we try and keep pace with the
>>    latest)
>> - OpenSSL has changed behaviour and we need to update our translation
>>    code to align with it (unusual)
>> - OpenSSL has changed behaviour and we need to update our tests to align
>>    with it (most frequent).
> Thank you both for the detailed explanation.  I suspected that I should
> had added the OpenSSL version to the OP.  On that Fedora machine I have
> OpenSSL 1.1.0i-fips 14 Aug 2018
> I tried the same tests on a Windows 10 machine.  Below are some
> discrepancies/peculiarities that I've noticed (I'd be happy to improve
> the test cases if possible):

I noticed some errors on Gump overnight so this morning I have build
OpenSSL 1.0.2, 1.1.0, 1.1.1 and master locally and tested them against
8.5.x and 9.0.x. I found a couple of bugs:

- The ARIA ciphers were not handled correctly so testing against OpenSSL
  1.1.0 was always going to fail. This has been fixed.

- 8.5.x was missing some code that ensured the OpenSSL libraries as well
  as the binary was on the path. This meant 8.5.x tests were either
  going to fail or use a locally installed OpenSSL version. This has
  also been fixed.

> On the Linux box I have OpenSSL installed and on the PATH.  On Windows I
> used version OpenSSL 1.1.1  11 Sep 2018 and specified it via the
> `test.openssl.path` property.  I checked the value of
> `test.openssl.exists` and it showed the expected `true`.  Both Windows
> and Fedora generated an output file for
> test/org/apache/tomcat/util/net/openssl/ [1]. Both,
> however, reported "Found OpenSSL version 0x0" which I find strange?

That does seem odd. I suspect either the wrong OpenSSL version or no
OpenSSL version was found.

> On Windows, only the output [2] for the file mentioned above is in the
> output/build/logs, while on Fedora I also have output from the 3 Test
> files from test/org/apache/tomcat/util/net/openssl/ciphers/. Does that
> mean that these tests were not run on Windows?

That seems to be a reasonable conclusion.

> I wanted to check the Gump output to compare with my local results. I
> found this URL, which I'm not sure if it is the right one or not -
> - as
> it says "Project build output found here..." but without any links or
> any other information.

That is the build. The full output is linked just below that line but
you probably want the tests which are run as a separate build for each

If you scroll down for any of those pages, you'll find the individual
test files for the latest run.

> I would like at the very least to add the output of `openssl version` to
> the Ant output, perhaps at the `test.openssl.exists` target.  If there
> are no objections I will add that.



To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message