zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Junqueira <...@apache.org>
Subject Re: Crypto Policy (was: Re: [VOTE] Apache ZooKeeper release 3.5.5 candidate 5)
Date Mon, 29 Apr 2019 14:53:52 GMT
I'm also +1 for adding a comment to the release notes (thanks for the suggestion, Ted). Updating
the readme makes sense, but the release notes will be the main source to indicate that we
require a specific or later version of Java from that particular release. My preference would
be to update the release notes.

As for running TLS on a single node, have you been able to do it? I haven't had a chance to
look further into it throughout my day, so if anyone has successfully done it and can share
some instructions, it would help me. Otherwise, I'll keep investigating once I have a chance.
To be specific, I created the keystore, certificate and truststore files according to instructions,
but the instructions assume that the aliases are different when it comes to populating the
truststore. At that point, I had to get creative and I have tried a couple of options that
didn't work. Either way, I think that being able to run locally and documenting is desirable,
although not a blocker. If I can get it right, then I can write a gist describing it that
we can use as a reference until we properly document it.

-Flavio

> On 29 Apr 2019, at 15:42, Andor Molnar <andor@apache.org> wrote:
> 
> Thanks Flavio for the investigation. I’ll update the README file to include instructions
on supported Java 8 versions.
> I’m wondering if I have to update the admin docs based on your problems running TLS
quorum on a single machine.
> 
> Andor
> 
> 
> 
>> On 2019. Apr 29., at 15:06, Enrico Olivelli <eolivelli@gmail.com> wrote:
>> 
>> Il lun 29 apr 2019, 13:44 Ted Dunning <ted.dunning@gmail.com> ha scritto:
>> 
>>> Other changes in u211+ substantially improve how Java 8 applications behave
>>> in containers. I am seeing this more and more with customers.
>>> 
>>> Combined with the crypto issues, it might be worth a release note
>>> suggesting that if you are going to compile with Java 1.8, you should use a
>>> recent release at u211 (u212?) Or above.
>>> 
>> 
>> +1 for a note on release docs
>> 
>> 
>> Enrico
>> 
>> 
>> 
>> 
>>> On Mon, Apr 29, 2019, 11:43 AM Flavio Junqueira <fpj@apache.org> wrote:
>>> 
>>>> I did a bit more research and it turns out that the crypto.policy option
>>>> was introduced u151:
>>>> 
>>>> 
>>> https://www.oracle.com/technetwork/java/javase/8u151-relnotes-3850493.html
>>>> <
>>>> 
>>> https://www.oracle.com/technetwork/java/javase/8u151-relnotes-3850493.html
>>>>> 
>>>> 
>>>> And started being defined by default with the "unlimited" option in u161:
>>>> 
>>>> 
>>> https://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html
>>>> <
>>>> 
>>> https://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html
>>>>> 
>>>> 
>>>> I have installed a more recent version, 1.8.0_211, and it builds fine
>>> (all
>>>> tests pass consistently for me).
>>>> 
>>>> 
>>>> I'm now trying to start an ensemble with ssl enabled locally, but it is
>>>> failing for me. It looks like the instructions in the admin doc assumes
>>>> different hosts. I need to look more closely into it to determine what
>>> not
>>>> is that I'm doing wrong, but in any case, the instructions do not make it
>>>> very clear whether one can run locally.
>>>> 
>>>> -Flavio
>>>> 
>>>>> On 27 Apr 2019, at 19:28, Patrick Hunt <phunt@apache.org> wrote:
>>>>> 
>>>>> Odd. I had done my testing on jdk11/macos which is fine.
>>>>> 
>>>>> I just tried jdk8 and 3 times in a row it's failing with:
>>>>> [ERROR]   SaslAuthTest.testZKOperationsAfterClientSaslAuthFailure:176
»
>>>>> Timeout Failed t...
>>>>> 
>>>>> I don't see the error Flavio is seeing. I have never installed special
>>>>> crypto libraries, etc... just vanilla jdk.
>>>>> 
>>>>> ⌂102% [phunt:~/Downloads/z/apache-zookeeper-3.5.5] 3s $ mvn --version
>>>>> Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555;
>>>>> 2019-04-04T12:00:29-07:00)
>>>>> Maven home: /usr/local/Cellar/maven/3.6.1/libexec
>>>>> Java version: 1.8.0_201, vendor: Oracle Corporation, runtime:
>>>>> /Library/Java/JavaVirtualMachines/jdk1.8.0_201.jdk/Contents/Home/jre
>>>>> Default locale: en_US, platform encoding: UTF-8
>>>>> OS name: "mac os x", version: "10.14.4", arch: "x86_64", family: "mac"
>>>>> 
>>>>> 
>>>>> 2019-04-27 10:11:51,635 [myid:] - INFO
>>>>> [NIOWorkerThread-6:SaslServerCallbackHandler@136] - Setting
>>>> authorizedID:
>>>>> super
>>>>> 2019-04-27 10:11:51,636 [myid:] - INFO
>>>>> [NIOWorkerThread-6:ZooKeeperServer@1170] - adding SASL authorization
>>> for
>>>>> authorizationID: super
>>>>> 2019-04-27 10:11:51,813 [myid:] - INFO
>>>>> [SessionTracker:SessionTrackerImpl@163] - SessionTrackerImpl exited
>>>> loop!
>>>>> 2019-04-27 10:12:21,596 [myid:] - INFO
>>>>> [main:JUnit4ZKTestRunner$LoggedInvokeMethod@98] - TEST METHOD FAILED
>>>>> testZKOperationsAfterClientSaslAuthFailure
>>>>> java.util.concurrent.TimeoutException: Failed to connect to ZooKeeper
>>>>> server.
>>>>> at
>>>>> 
>>>> 
>>> org.apache.zookeeper.test.ClientBase$CountdownWatcher.waitForConnected(ClientBase.java:150)
>>>>> at
>>>>> 
>>>> 
>>> org.apache.zookeeper.SaslAuthTest.testZKOperationsAfterClientSaslAuthFailure(SaslAuthTest.java:176)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at
>>>>> 
>>>> 
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>>> at
>>>>> 
>>>> 
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>>> at
>>>>> 
>>>> 
>>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>>>>> at
>>>>> 
>>>> 
>>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>>>>> 
>>>>> 
>>>>> Patrick
>>>>> 
>>>>> 
>>>>> On Sat, Apr 27, 2019 at 8:54 AM Enrico Olivelli <eolivelli@gmail.com>
>>>> wrote:
>>>>> 
>>>>>> On my local tests I usually don't get the error because I am using
>>>>>> jdk11 and unlimited strength cryptography is enabled by default
>>>>>> 
>>>>>> 
>>>> 
>>> https://www.oracle.com/technetwork/java/javase/documentation/jdk11-readme-5097204.html#jce
>>>>>> 
>>>>>> In production it depends on the configuration of SSL, if you require
>>>>>> strong ciphers/big keys you will have problems, but the server won't
>>>>>> start so you will find soon the problem.
>>>>>> I think this is not a real issue (for production I mean).
>>>>>> I see these ways:
>>>>>> 1) adapt the tests in order to make default jdk8 happy
>>>>>> 2) tweak the tests enabling "unlimited strenght cryptography" using
>>>>>> reflection
>>>>>> 3) just write a line in documentation that says that in order to
make
>>>>>> tests pass you have to enable the flag
>>>>>> 
>>>>>> That flag is deprecated and enabled by default in modern JREs, so
I
>>>>>> would go for 2) or 3)
>>>>>> 
>>>>>> I guess that on  ASF Jenkins if the JDK8 we are using has the flag
>>>> turned
>>>>>> on
>>>>>> 
>>>>>> Enrico
>>>>>> 
>>>>>> Il giorno sab 27 apr 2019 alle ore 17:48 Andor Molnar
>>>>>> <andor@apache.org> ha scritto:
>>>>>>> 
>>>>>>> I’m running the tests fine without setting the policy to unlimited:
>>>>>>> 
>>>>>>> java version "1.8.0_161"
>>>>>>> Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
>>>>>>> Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)
>>>>>>> 
>>>>>>> Have you tried to run it with a more recent version of Java?
>>>>>>> 
>>>>>>> Andor
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> On 2019. Apr 27., at 17:33, Andor Molnar <andor@apache.org>
wrote:
>>>>>>>> 
>>>>>>>> Good catch, thanks Flavio for reporting this. We need to
double
>>> check
>>>>>> the tests with Ilya I believe.
>>>>>>>> 
>>>>>>>> Having tests failure means that you were actually able to
_build_
>>>>>> ZooKeeper successfully without changing the crypto policy setting.
>>> Have
>>>> you
>>>>>> tried to start an ensemble with Quorum TLS by any chance? That would
>>> add
>>>>>> some more color to this issue.
>>>>>>>> 
>>>>>>>> This might be just a testing issue.
>>>>>>>> 
>>>>>>>> Regards,
>>>>>>>> Andor
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> On 2019. Apr 27., at 16:09, Flavio Junqueira <fpj@apache.org>
>>> wrote:
>>>>>>>>> 
>>>>>>>>> Hi Enrico,
>>>>>>>>> 
>>>>>>>>> Here is the info you are requesting:
>>>>>>>>> 
>>>>>>>>> *Java version*
>>>>>>>>> 
>>>>>>>>> $ java -version
>>>>>>>>> java version "1.8.0_152"
>>>>>>>>> Java(TM) SE Runtime Environment (build 1.8.0_152-b16)
>>>>>>>>> Java HotSpot(TM) 64-Bit Server VM (build 25.152-b16,
mixed mode)
>>>>>>>>> 
>>>>>>>>> *Test case errors*
>>>>>>>>> 
>>>>>>>>> I won’t post all of them, I get a good number of errors:
>>>>>>>>> 
>>>>>>>>> ================================
>>>>>>>>> [ERROR] Tests run: 64, Failures: 0, Errors: 16, Skipped:
0, Time
>>>>>> elapsed: 9.21 s <<< FAILURE! - in
>>>> org.apache.zookeeper.util.PemReaderTest
>>>>>>>>> [ERROR]
>>>>>> 
>>>> 
>>> testLoadCertificateFromKeyStore[1](org.apache.zookeeper.util.PemReaderTest)
>>>>>> Time elapsed: 1.593 s  <<< ERROR!
>>>>>>>>> java.io.IOException:
>>>>>> org.bouncycastle.operator.OperatorCreationException: Illegal key
size
>>> or
>>>>>> default parameters
>>>>>>>>>  at
>>>>>> 
>>>> 
>>> org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(PemReaderTest.java:125)
>>>>>>>>> Caused by: org.bouncycastle.operator.OperatorCreationException:
>>>>>> Illegal key size or default parameters
>>>>>>>>>  at
>>>>>> 
>>>> 
>>> org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(PemReaderTest.java:125)
>>>>>>>>> Caused by: java.security.InvalidKeyException: Illegal
key size or
>>>>>> default parameters
>>>>>>>>>  at
>>>>>> 
>>>> 
>>> org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(PemReaderTest.java:125)
>>>>>>>>> 
>>>>>>>>> [ERROR]
>>>>>> 
>>>> 
>>> testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword[1](org.apache.zookeeper.util.PemReaderTest)
>>>>>> Time elapsed: 0.004 s  <<< ERROR!
>>>>>>>>> java.lang.Exception: Unexpected exception,
>>>>>> expected<java.security.GeneralSecurityException> but
>>>>>> was<java.io.IOException>
>>>>>>>>>  at
>>>>>> 
>>>> 
>>> org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword(PemReaderTest.java:93)
>>>>>>>>> Caused by: org.bouncycastle.operator.OperatorCreationException:
>>>>>> Illegal key size or default parameters
>>>>>>>>>  at
>>>>>> 
>>>> 
>>> org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword(PemReaderTest.java:93)
>>>>>>>>> Caused by: java.security.InvalidKeyException: Illegal
key size or
>>>>>> default parameters
>>>>>>>>>  at
>>>>>> 
>>>> 
>>> org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword(PemReaderTest.java:93)
>>>>>>>>> ...
>>>>>>>>> ================================
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> *Crypto policy*
>>>>>>>>> If I uncomment this configuration option:
>>>>>>>>> 
>>>>>>>>> # Please see the JCA documentation for additional information
on
>>>> these
>>>>>>>>> # files and formats.
>>>>>>>>> # crypto.policy=unlimited
>>>>>>>>> 
>>>>>>>>> in:
>>>>>>>>> 
>>>>>>>>> $JAVA_HOME/jre/lib/security/java.security
>>>>>>>>> 
>>>>>>>>> then it all works and I get no error at all. This option
controls
>>>>>> cryptographic strengths according to the documentation, and is present
>>>>>> because of crypto regulations in different countries.
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> -Flavio
>>>>>>>>> 
>>>>>>>>>> On 27 Apr 2019, at 15:52, Enrico Olivelli <eolivelli@gmail.com>
>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Il sab 27 apr 2019, 14:18 Flavio Junqueira <fpj@apache.org>
ha
>>>>>> scritto:
>>>>>>>>>> 
>>>>>>>>>>> I have a clarification question about the RC.
To build the RC, I
>>>>>> had to
>>>>>>>>>>> enable crypto.policy unlimited in the jre (I'm
using build
>>>>>> 1.8.0_152-b16).
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> Flavio
>>>>>>>>>> What do you mean with 'build' ?
>>>>>>>>>> Make tests pass?
>>>>>>>>>> AFAIK we are not using tweaked jdks in CI builds,
so in theory
>>> there
>>>>>> is no
>>>>>>>>>> need.
>>>>>>>>>> 
>>>>>>>>>> Can you please share your error?
>>>>>>>>>> 
>>>>>>>>>> Enrico
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> I'm wondering if this is going to be an issue for
some users as
>>> this
>>>>>> option
>>>>>>>>>>> is related to import/export regulation. Has anyone
looked into it
>>>>>> and could
>>>>>>>>>>> clarify it to me, please?
>>>>>>>>>>> 
>>>>>>>>>>> Thanks,
>>>>>>>>>>> -Flavio
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>> On 25 Apr 2019, at 15:10, Andor Molnar <andor@apache.org>
>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> This is the first stable release of 3.5 branch:
3.5.5. It
>>> resolves
>>>>>> 117
>>>>>>>>>>> issues, including Maven migration, Quorum TLS,
TTL nodes and lots
>>>>>> of other
>>>>>>>>>>> performance and stability improvements.
>>>>>>>>>>>> 
>>>>>>>>>>>> The full release notes is available at:
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>> 
>>>> 
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801&version=12343268
>>>>>>>>>>>> 
>>>>>>>>>>>> *** Please download, test and vote by May
3rd 2019, 23:59 UTC+0.
>>>>>> ***
>>>>>>>>>>>> 
>>>>>>>>>>>> Source files:
>>>>>>>>>>>> 
>>>>>> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.5.5-rc5/
>>>>>>>>>>>> 
>>>>>>>>>>>> Maven staging repos:
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>> 
>>>> 
>>> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/parent/3.5.5/
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>> 
>>>> 
>>> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper-jute/3.5.5/
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>> 
>>>> 
>>> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.5/
>>>>>>>>>>>> 
>>>>>>>>>>>> The release candidate tag in git to be voted
upon:
>>>>>> release-3.5.5-rc5
>>>>>>>>>>>> 
>>>>>>>>>>>> ZooKeeper's KEYS file containing PGP keys
we use to sign the
>>>>>> release:
>>>>>>>>>>>> http://www.apache.org/dist/zookeeper/KEYS
>>>>>>>>>>>> 
>>>>>>>>>>>> Should we release this candidate?
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> 
>>>> 
>>> 
> 


Mime
View raw message