tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Schultz <ch...@christopherschultz.net>
Subject Re: [OT] Tomcat 7.0.55/Jre 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac
Date Tue, 13 Oct 2015 13:42:15 GMT
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

George,

On 10/9/15 12:46 PM, George Stanchev wrote:
> One more clarification: on point [6] below I stated that Java is
> able to recover with a retry on a cached connection. Unfortunately
> that is only valid for higher level classes like HttpUrlConnection
> which makes 1 retry on IOException (and only on a GET and some
> other conditions). The lower-level sockets just throw and that’s
> it...

Can you confirm whether using SSLv2Hello also solves the problem? Or
do you have a requirement that nothing less than TLSv1 may even
handshake properly?

- -chris

> -----Original Message----- From: George Stanchev
> [mailto:Gstanchev@serena.com] Sent: Friday, October 09, 2015 10:40
> AM To: Tomcat Users List Subject: RE: [OT] Tomcat 7.0.55/Jre 7u67:
> SEND TLSv1 ALERT: fatal, description = bad_record_mac
> 
> Just for the record, https.protocols is a property used by the
> HttpsUrlConnection class. If your app is using a client that
> doesn't rely on the internal Oracle HTTP client, it's better to use
> " jdk.tls.client.protocols" which is read directly by the
> socket/SSL classes. Apache Http Client is one client that does use
> sockets directly instead of HttpsUrlConnection.
> 
> 
> Also, I want to mention that we have finally been able to isolate a
> really nasty problem with Java SSL implementation and Windows's
> SSPI based SChannel acting as a server. The issue we ran into is
> very similar as to what the OP reported, that’s why I asked him a
> while ago on a separate thread about keystores if the server is
> IIS-based. The issue that we ran into might sound esoteric but our
> product ran into it very often. We have a configuration where we
> lock our intracomponent communication via 2way SSL. We have Java
> Tomcat-based server components that talk to IIS based native
> components via web services over HTTP which we secure with 2way
> SSL. When we upgraded to JRE 8 we started getting those connection
> resets in Tomcat when we enabled the 2way SSL. Those were on the
> Java->IIS calls that work over regular SSL. On the java side, if
> you turn on the ssl debug on, you'd see connection reset
> IOException thrown, and on the Windows side, you'd see " The
> following fatal alert was generated: 20. The internal error state
> is 960." In the System event log. This error is "
> TLS1_ALERT_BAD_RECORD_MAC" from SSPI. A search in Oracle's bug
> database found a close resemblance to this issue:
> https://bugs.openjdk.java.net/browse/JDK-8068812. But the reporter
> couldn't reproduce it and also was against another Oracle component
> Java Web Download or something like this. So we started working
> with Oracle but the intermittent and hard-to-reproduce nature of
> this issue hampered the investigation. As the OP mentioned below,
> there are apocryphal reports for this all over the web but people
> cannot troubleshoot it very well. Anyway here are my findings. In
> order for the issue to arise all those conditions must be met:
> 
> [1] The client must start with ClientHello TLSv1.2 [2] The server
> must respond with ServerHello TLSv1 (so the server should not have
> TLS1.2 enabled) [3] The cipher key exchange must NOT be ECDHE. I
> have not tried DHE as I could not make Windows take it. According
> to this doc [1] it should have several DHE key exchange ciphers
> available but I could not make it work. RSA key exchange exhibits
> the issue. I have been using “TLS_RSA_WITH_AES_256_CBC_SHA” to
> reproduce. [4] The initial connection must not be 2way-SSL [5] The
> server must upgrade the connection after encrypted payload is read.
> In case of IIS, it has 2way SSL setting on a Virtual Directory. The
> initial handshake is not 2way SSL, once the HTTP request is read,
> it determines that the requested resource (/gsoap) is 2way
> SSL-protected and upgrades the connection to 2way SSL. [6] The
> server must kill the connection abruptly when error happens. This
> is important as if the server terminates the connection gracefully
> and keeps it around Java is able to recover since it reuses the
> cached connection. IIS does exactly this. [7] The server must be
> Windows 7, Win2k8, Win2k8-R2, Win8. Might be other versions
> affected but bug is not evident on Win2k12 or Win2k12-R2 or Win10.
> Perhaps it was fixed, perhaps those versions support TLSv1.2
> 
> It might sound complicated but it is not, it is very common. All
> you have to do is to have a Virtual Directory in IIS that requires
> 2way SSL and is set to "want". RSA-based key exchanges are one of
> the most commonly used ciphers. The Windows platforms we tested
> with are still widely used in the enterprises while the adoption
> rate of win2k12+ is still lagging (from our experience with our
> customers). And you have equal "chance" of cipher support between
> Windows servers and Java 8 clients to land on RSA based cipher
> (which exhibits the issue) or ECDHE key exchange that is ok. Again,
> it might sound too complicated but it was blocker for us to ship
> our product.
> 
> For us, to work around this issue is for the 2way SSL to turn off
> TLSv1.2 (via "-Djdk.tls.client.protocols=TLSv1" or RSA key exchange
> algorithm (via the java security properties) on affected systems
> which fixes it.
> 
> I have a lot more technical details that I am not sharing here. I
> have sent them to Oracle so hopefully that gets resolved, but my
> guess is that is really an SSPI/SChannel issue in which case
> someone needs to work with Microsoft and I don't know if my company
> can do that. It boils down to - when all these conditions are met,
> is Java creating a bad MAC or SChannel fails to verify that MAC
> correctly on SSL handshake step X.
> 
> Hopefully this can help someone else, as we have spent quite a bit
> of time on this issue.
> 
> George
> 
> 
> TL;DR: There is a nasty bug in Java or Windows when calling
> SSPI-based app over 2way SSL that you can work around by turning
> off TLSv1.2 or RSA key exchanges.
> 
> -----Original Message----- From: Christopher Schultz
> [mailto:chris@christopherschultz.net] Sent: Monday, August 31, 2015
> 11:15 AM To: Tomcat Users List Subject: Re: [OT] Tomcat 7.0.55/Jre
> 7u67: SEND TLSv1 ALERT: fatal, description = bad_record_mac
> 
> Diarmuid,
> 
> On 8/31/15 11:36 AM, dmccrthy wrote:
>> To cut a long story short openssl helped. Using openssl -connect
>>  showed the error below. When I added the -tls1 flag the error
>> went away.
> 
>> 2104:error:1408F119:SSL routines:SSL3_GET_RECORD:decryption
>> failed or bad record mac:.\ssl\s3_pkt.c:532:
> 
>> Configuring Tomcat with the JVM parameter below resolved the
>> issue.
> 
>> -Dhttps.protocols="TLSv1"
> 
> Yep, you have to use a TLS handshake; the older SSL3 handshake
> won't wor k.
> 
> Now... you *can* enable SSLv2Hello and then only support TLSv1+,
> but let's face it: SSL is dead and clients should get with the
> program.
> 
>> This may limit our client Web application but I have very little
>>  influence over the server-side application, so it'll have to do
>> as a solution for now.
> 
> Your client should not try to use SSL handshake unless you know
> it's required.
> 
>> So the issue seems to be a Java handshake error whereby it has a
>>  problem downgrading from TLSv1.2 to TLSv1 during the handshake.
> 
> I don't know enough about TLS to know if there is an appreciable
> difference between the handshake protocol between the two. I know
> that SSLv2 was the lowest common denominator for a while, and most
> servers would enable it *just for the handshake* but would refuse
> to actually negotiate an SSLv2 connection (SSLv2 was basically
> DOA). These days, nobody should be using SSL at all, but I might
> understand wanting to use SSL-compatible handshakes, so using
> "SSLv2Hello" but not supporting the SSLv3 protocol would be the way
> to do that.
> 
> Again, we should just let SSL die. The more we push clients and
> servers to only negotiate TLS from here on out, the better off
> Internet security will be overall.
> 
>> Why it works in our non-prod environment with the same versions
>> of everything is a mystery, but I can speculate at some
>> difference in underlying O/S settings, or perhaps a Java bug.
> 
> Or more likely a configuration option you didn't detect when you
> checked everything out.
> 
> -chris
> 
> ---------------------------------------------------------------------
>
> 
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
> 
> 
> ---------------------------------------------------------------------
>
> 
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
> 
> 
> ---------------------------------------------------------------------
>
> 
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
> 
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iQIcBAEBCAAGBQJWHQo3AAoJEBzwKT+lPKRYyikQAIrCIi/xUNf6MBzJAGT1zLk0
lmxNVDcve7NrLcESBWXlCZl2iDNrDS1jxqLG8Nv6aZu64Jkz4tRh2b+nGXIXzrhc
NCejO7t4bWqqrucsjjT5SDJn2GCJEz2+usMvWTUXH+YUhJNRuXC/6xAjSP5W3vlO
TffM5U0kiQFHUWfHrcn2kuVWgvD7uA9oSE4vhb7qilypWR+0F0jw6BQOSkzpg9S7
zT3Pe2Mpl5M0au+zqDP9kQTk3E784OAyinfVz7lWsh8hd4ZdFz0cDr3SZSpFaMXI
po2yf2I8lF1ocrlC7fG7PDwtDnBRLVTo/ZB7hYHG9yWvWzp96KzHi+IZ1QfpjozR
oqL4SJFZmREAGZ17ry8AT3spp2L3Axp7gVBX+mkseUhZwAesf18OgPUlgvXaLkG2
yy2BYmiGwfyyItg+ipgH6XmOl0iptOTy8wrg3rtNwlzX21ZrhdRRWprjdDc550jp
Bt+MJw7YiLDyG5F4gSlXe2PI9PrpJ7/Ed2+vW6noJGPaPMnjctksCSH6jl60SUmn
0m9v8jrGxqTV/0ib+fkgIdmJZE7dMPQkvMQG+Skhx0H2MObJeM30ILGFSS8J17hi
W29muAGCuD5cy1PTfivRDxQZf8JHVjZ6wmUC2h4NclTamPClnqPxtXW7AmDwkCzj
Nfynh7OSvHG2bJdaFGYI
=8/jo
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message