zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: Crypto Policy (was: Re: [VOTE] Apache ZooKeeper release 3.5.5 candidate 5)
Date Mon, 29 Apr 2019 11:38:50 GMT
See https://royvanrijn.com/blog/2018/05/java-and-docker-memory-limits/ for
more info on the problem I was referring to.



On Mon, Apr 29, 2019, 12:36 PM Ted Dunning <ted.dunning@gmail.com> wrote:

> Other changes in u211+ substantially improve how Java 8 applications
> behave in containers. I am seeing this more and more with customers.
>
> Combined with the crypto issues, it might be worth a release note
> suggesting that if you are going to compile with Java 1.8, you should use a
> recent release at u211 (u212?) Or above.
>
> On Mon, Apr 29, 2019, 11:43 AM Flavio Junqueira <fpj@apache.org> wrote:
>
>> I did a bit more research and it turns out that the crypto.policy option
>> was introduced u151:
>>
>> https://www.oracle.com/technetwork/java/javase/8u151-relnotes-3850493.html
>> <
>> https://www.oracle.com/technetwork/java/javase/8u151-relnotes-3850493.html
>> >
>>
>> And started being defined by default with the "unlimited" option in u161:
>>
>> https://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html
>> <
>> https://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html
>> >
>>
>> I have installed a more recent version, 1.8.0_211, and it builds fine
>> (all tests pass consistently for me).
>>
>>
>> I'm now trying to start an ensemble with ssl enabled locally, but it is
>> failing for me. It looks like the instructions in the admin doc assumes
>> different hosts. I need to look more closely into it to determine what not
>> is that I'm doing wrong, but in any case, the instructions do not make it
>> very clear whether one can run locally.
>>
>> -Flavio
>>
>> > On 27 Apr 2019, at 19:28, Patrick Hunt <phunt@apache.org> wrote:
>> >
>> > Odd. I had done my testing on jdk11/macos which is fine.
>> >
>> > I just tried jdk8 and 3 times in a row it's failing with:
>> > [ERROR]   SaslAuthTest.testZKOperationsAfterClientSaslAuthFailure:176 »
>> > Timeout Failed t...
>> >
>> > I don't see the error Flavio is seeing. I have never installed special
>> > crypto libraries, etc... just vanilla jdk.
>> >
>> > ⌂102% [phunt:~/Downloads/z/apache-zookeeper-3.5.5] 3s $ mvn --version
>> > Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555;
>> > 2019-04-04T12:00:29-07:00)
>> > Maven home: /usr/local/Cellar/maven/3.6.1/libexec
>> > Java version: 1.8.0_201, vendor: Oracle Corporation, runtime:
>> > /Library/Java/JavaVirtualMachines/jdk1.8.0_201.jdk/Contents/Home/jre
>> > Default locale: en_US, platform encoding: UTF-8
>> > OS name: "mac os x", version: "10.14.4", arch: "x86_64", family: "mac"
>> >
>> >
>> > 2019-04-27 10:11:51,635 [myid:] - INFO
>> > [NIOWorkerThread-6:SaslServerCallbackHandler@136] - Setting
>> authorizedID:
>> > super
>> > 2019-04-27 10:11:51,636 [myid:] - INFO
>> > [NIOWorkerThread-6:ZooKeeperServer@1170] - adding SASL authorization
>> for
>> > authorizationID: super
>> > 2019-04-27 10:11:51,813 [myid:] - INFO
>> > [SessionTracker:SessionTrackerImpl@163] - SessionTrackerImpl exited
>> loop!
>> > 2019-04-27 10:12:21,596 [myid:] - INFO
>> > [main:JUnit4ZKTestRunner$LoggedInvokeMethod@98] - TEST METHOD FAILED
>> > testZKOperationsAfterClientSaslAuthFailure
>> > java.util.concurrent.TimeoutException: Failed to connect to ZooKeeper
>> > server.
>> > at
>> >
>> org.apache.zookeeper.test.ClientBase$CountdownWatcher.waitForConnected(ClientBase.java:150)
>> > at
>> >
>> org.apache.zookeeper.SaslAuthTest.testZKOperationsAfterClientSaslAuthFailure(SaslAuthTest.java:176)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > at
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> > at
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > at java.lang.reflect.Method.invoke(Method.java:498)
>> > at
>> >
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>> > at
>> >
>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>> >
>> >
>> > Patrick
>> >
>> >
>> > On Sat, Apr 27, 2019 at 8:54 AM Enrico Olivelli <eolivelli@gmail.com>
>> wrote:
>> >
>> >> On my local tests I usually don't get the error because I am using
>> >> jdk11 and unlimited strength cryptography is enabled by default
>> >>
>> >>
>> https://www.oracle.com/technetwork/java/javase/documentation/jdk11-readme-5097204.html#jce
>> >>
>> >> In production it depends on the configuration of SSL, if you require
>> >> strong ciphers/big keys you will have problems, but the server won't
>> >> start so you will find soon the problem.
>> >> I think this is not a real issue (for production I mean).
>> >> I see these ways:
>> >> 1) adapt the tests in order to make default jdk8 happy
>> >> 2) tweak the tests enabling "unlimited strenght cryptography" using
>> >> reflection
>> >> 3) just write a line in documentation that says that in order to make
>> >> tests pass you have to enable the flag
>> >>
>> >> That flag is deprecated and enabled by default in modern JREs, so I
>> >> would go for 2) or 3)
>> >>
>> >> I guess that on  ASF Jenkins if the JDK8 we are using has the flag
>> turned
>> >> on
>> >>
>> >> Enrico
>> >>
>> >> Il giorno sab 27 apr 2019 alle ore 17:48 Andor Molnar
>> >> <andor@apache.org> ha scritto:
>> >>>
>> >>> I’m running the tests fine without setting the policy to unlimited:
>> >>>
>> >>> java version "1.8.0_161"
>> >>> Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
>> >>> Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)
>> >>>
>> >>> Have you tried to run it with a more recent version of Java?
>> >>>
>> >>> Andor
>> >>>
>> >>>
>> >>>
>> >>>> On 2019. Apr 27., at 17:33, Andor Molnar <andor@apache.org>
wrote:
>> >>>>
>> >>>> Good catch, thanks Flavio for reporting this. We need to double
check
>> >> the tests with Ilya I believe.
>> >>>>
>> >>>> Having tests failure means that you were actually able to _build_
>> >> ZooKeeper successfully without changing the crypto policy setting.
>> Have you
>> >> tried to start an ensemble with Quorum TLS by any chance? That would
>> add
>> >> some more color to this issue.
>> >>>>
>> >>>> This might be just a testing issue.
>> >>>>
>> >>>> Regards,
>> >>>> Andor
>> >>>>
>> >>>>
>> >>>>
>> >>>>> On 2019. Apr 27., at 16:09, Flavio Junqueira <fpj@apache.org>
>> wrote:
>> >>>>>
>> >>>>> Hi Enrico,
>> >>>>>
>> >>>>> Here is the info you are requesting:
>> >>>>>
>> >>>>> *Java version*
>> >>>>>
>> >>>>> $ java -version
>> >>>>> java version "1.8.0_152"
>> >>>>> Java(TM) SE Runtime Environment (build 1.8.0_152-b16)
>> >>>>> Java HotSpot(TM) 64-Bit Server VM (build 25.152-b16, mixed mode)
>> >>>>>
>> >>>>> *Test case errors*
>> >>>>>
>> >>>>> I won’t post all of them, I get a good number of errors:
>> >>>>>
>> >>>>> ================================
>> >>>>> [ERROR] Tests run: 64, Failures: 0, Errors: 16, Skipped: 0,
Time
>> >> elapsed: 9.21 s <<< FAILURE! - in
>> org.apache.zookeeper.util.PemReaderTest
>> >>>>> [ERROR]
>> >>
>> testLoadCertificateFromKeyStore[1](org.apache.zookeeper.util.PemReaderTest)
>> >> Time elapsed: 1.593 s  <<< ERROR!
>> >>>>> java.io.IOException:
>> >> org.bouncycastle.operator.OperatorCreationException: Illegal key size
>> or
>> >> default parameters
>> >>>>>     at
>> >>
>> org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(PemReaderTest.java:125)
>> >>>>> Caused by: org.bouncycastle.operator.OperatorCreationException:
>> >> Illegal key size or default parameters
>> >>>>>     at
>> >>
>> org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(PemReaderTest.java:125)
>> >>>>> Caused by: java.security.InvalidKeyException: Illegal key size
or
>> >> default parameters
>> >>>>>     at
>> >>
>> org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(PemReaderTest.java:125)
>> >>>>>
>> >>>>> [ERROR]
>> >>
>> testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword[1](org.apache.zookeeper.util.PemReaderTest)
>> >> Time elapsed: 0.004 s  <<< ERROR!
>> >>>>> java.lang.Exception: Unexpected exception,
>> >> expected<java.security.GeneralSecurityException> but
>> >> was<java.io.IOException>
>> >>>>>     at
>> >>
>> org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword(PemReaderTest.java:93)
>> >>>>> Caused by: org.bouncycastle.operator.OperatorCreationException:
>> >> Illegal key size or default parameters
>> >>>>>     at
>> >>
>> org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword(PemReaderTest.java:93)
>> >>>>> Caused by: java.security.InvalidKeyException: Illegal key size
or
>> >> default parameters
>> >>>>>     at
>> >>
>> org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword(PemReaderTest.java:93)
>> >>>>> ...
>> >>>>> ================================
>> >>>>>
>> >>>>>
>> >>>>> *Crypto policy*
>> >>>>> If I uncomment this configuration option:
>> >>>>>
>> >>>>> # Please see the JCA documentation for additional information
on
>> these
>> >>>>> # files and formats.
>> >>>>> # crypto.policy=unlimited
>> >>>>>
>> >>>>> in:
>> >>>>>
>> >>>>> $JAVA_HOME/jre/lib/security/java.security
>> >>>>>
>> >>>>> then it all works and I get no error at all. This option controls
>> >> cryptographic strengths according to the documentation, and is present
>> >> because of crypto regulations in different countries.
>> >>>>>
>> >>>>> Thanks,
>> >>>>> -Flavio
>> >>>>>
>> >>>>>> On 27 Apr 2019, at 15:52, Enrico Olivelli <eolivelli@gmail.com>
>> >> wrote:
>> >>>>>>
>> >>>>>> Il sab 27 apr 2019, 14:18 Flavio Junqueira <fpj@apache.org>
ha
>> >> scritto:
>> >>>>>>
>> >>>>>>> I have a clarification question about the RC. To build
the RC, I
>> >> had to
>> >>>>>>> enable crypto.policy unlimited in the jre (I'm using
build
>> >> 1.8.0_152-b16).
>> >>>>>>
>> >>>>>>
>> >>>>>> Flavio
>> >>>>>> What do you mean with 'build' ?
>> >>>>>> Make tests pass?
>> >>>>>> AFAIK we are not using tweaked jdks in CI builds, so in
theory
>> there
>> >> is no
>> >>>>>> need.
>> >>>>>>
>> >>>>>> Can you please share your error?
>> >>>>>>
>> >>>>>> Enrico
>> >>>>>>
>> >>>>>>
>> >>>>>> I'm wondering if this is going to be an issue for some users
as
>> this
>> >> option
>> >>>>>>> is related to import/export regulation. Has anyone looked
into it
>> >> and could
>> >>>>>>> clarify it to me, please?
>> >>>>>>>
>> >>>>>>> Thanks,
>> >>>>>>> -Flavio
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> On 25 Apr 2019, at 15:10, Andor Molnar <andor@apache.org>
wrote:
>> >>>>>>>>
>> >>>>>>>> This is the first stable release of 3.5 branch:
3.5.5. It
>> resolves
>> >> 117
>> >>>>>>> issues, including Maven migration, Quorum TLS, TTL nodes
and lots
>> >> of other
>> >>>>>>> performance and stability improvements.
>> >>>>>>>>
>> >>>>>>>> The full release notes is available at:
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801&version=12343268
>> >>>>>>>>
>> >>>>>>>> *** Please download, test and vote by May 3rd 2019,
23:59 UTC+0.
>> >> ***
>> >>>>>>>>
>> >>>>>>>> Source files:
>> >>>>>>>>
>> >> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.5.5-rc5/
>> >>>>>>>>
>> >>>>>>>> Maven staging repos:
>> >>>>>>>>
>> >>>>>>>
>> >>
>> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/parent/3.5.5/
>> >>>>>>>>
>> >>>>>>>
>> >>
>> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper-jute/3.5.5/
>> >>>>>>>>
>> >>>>>>>
>> >>
>> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.5/
>> >>>>>>>>
>> >>>>>>>> The release candidate tag in git to be voted upon:
>> >> release-3.5.5-rc5
>> >>>>>>>>
>> >>>>>>>> ZooKeeper's KEYS file containing PGP keys we use
to sign the
>> >> release:
>> >>>>>>>> http://www.apache.org/dist/zookeeper/KEYS
>> >>>>>>>>
>> >>>>>>>> Should we release this candidate?
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>
>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message