zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Enrico Olivelli <eolive...@gmail.com>
Subject Re: Crypto Policy (was: Re: [VOTE] Apache ZooKeeper release 3.5.5 candidate 5)
Date Mon, 29 Apr 2019 13:06:20 GMT
Il lun 29 apr 2019, 13:44 Ted Dunning <ted.dunning@gmail.com> ha scritto:

> Other changes in u211+ substantially improve how Java 8 applications behave
> in containers. I am seeing this more and more with customers.
>
> Combined with the crypto issues, it might be worth a release note
> suggesting that if you are going to compile with Java 1.8, you should use a
> recent release at u211 (u212?) Or above.
>

+1 for a note on release docs


Enrico




> On Mon, Apr 29, 2019, 11:43 AM Flavio Junqueira <fpj@apache.org> wrote:
>
> > I did a bit more research and it turns out that the crypto.policy option
> > was introduced u151:
> >
> >
> https://www.oracle.com/technetwork/java/javase/8u151-relnotes-3850493.html
> > <
> >
> https://www.oracle.com/technetwork/java/javase/8u151-relnotes-3850493.html
> > >
> >
> > And started being defined by default with the "unlimited" option in u161:
> >
> >
> https://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html
> > <
> >
> https://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html
> > >
> >
> > I have installed a more recent version, 1.8.0_211, and it builds fine
> (all
> > tests pass consistently for me).
> >
> >
> > I'm now trying to start an ensemble with ssl enabled locally, but it is
> > failing for me. It looks like the instructions in the admin doc assumes
> > different hosts. I need to look more closely into it to determine what
> not
> > is that I'm doing wrong, but in any case, the instructions do not make it
> > very clear whether one can run locally.
> >
> > -Flavio
> >
> > > On 27 Apr 2019, at 19:28, Patrick Hunt <phunt@apache.org> wrote:
> > >
> > > Odd. I had done my testing on jdk11/macos which is fine.
> > >
> > > I just tried jdk8 and 3 times in a row it's failing with:
> > > [ERROR]   SaslAuthTest.testZKOperationsAfterClientSaslAuthFailure:176 »
> > > Timeout Failed t...
> > >
> > > I don't see the error Flavio is seeing. I have never installed special
> > > crypto libraries, etc... just vanilla jdk.
> > >
> > > ⌂102% [phunt:~/Downloads/z/apache-zookeeper-3.5.5] 3s $ mvn --version
> > > Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555;
> > > 2019-04-04T12:00:29-07:00)
> > > Maven home: /usr/local/Cellar/maven/3.6.1/libexec
> > > Java version: 1.8.0_201, vendor: Oracle Corporation, runtime:
> > > /Library/Java/JavaVirtualMachines/jdk1.8.0_201.jdk/Contents/Home/jre
> > > Default locale: en_US, platform encoding: UTF-8
> > > OS name: "mac os x", version: "10.14.4", arch: "x86_64", family: "mac"
> > >
> > >
> > > 2019-04-27 10:11:51,635 [myid:] - INFO
> > > [NIOWorkerThread-6:SaslServerCallbackHandler@136] - Setting
> > authorizedID:
> > > super
> > > 2019-04-27 10:11:51,636 [myid:] - INFO
> > > [NIOWorkerThread-6:ZooKeeperServer@1170] - adding SASL authorization
> for
> > > authorizationID: super
> > > 2019-04-27 10:11:51,813 [myid:] - INFO
> > > [SessionTracker:SessionTrackerImpl@163] - SessionTrackerImpl exited
> > loop!
> > > 2019-04-27 10:12:21,596 [myid:] - INFO
> > > [main:JUnit4ZKTestRunner$LoggedInvokeMethod@98] - TEST METHOD FAILED
> > > testZKOperationsAfterClientSaslAuthFailure
> > > java.util.concurrent.TimeoutException: Failed to connect to ZooKeeper
> > > server.
> > > at
> > >
> >
> org.apache.zookeeper.test.ClientBase$CountdownWatcher.waitForConnected(ClientBase.java:150)
> > > at
> > >
> >
> org.apache.zookeeper.SaslAuthTest.testZKOperationsAfterClientSaslAuthFailure(SaslAuthTest.java:176)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > > at
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > at java.lang.reflect.Method.invoke(Method.java:498)
> > > at
> > >
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> > > at
> > >
> >
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> > >
> > >
> > > Patrick
> > >
> > >
> > > On Sat, Apr 27, 2019 at 8:54 AM Enrico Olivelli <eolivelli@gmail.com>
> > wrote:
> > >
> > >> On my local tests I usually don't get the error because I am using
> > >> jdk11 and unlimited strength cryptography is enabled by default
> > >>
> > >>
> >
> https://www.oracle.com/technetwork/java/javase/documentation/jdk11-readme-5097204.html#jce
> > >>
> > >> In production it depends on the configuration of SSL, if you require
> > >> strong ciphers/big keys you will have problems, but the server won't
> > >> start so you will find soon the problem.
> > >> I think this is not a real issue (for production I mean).
> > >> I see these ways:
> > >> 1) adapt the tests in order to make default jdk8 happy
> > >> 2) tweak the tests enabling "unlimited strenght cryptography" using
> > >> reflection
> > >> 3) just write a line in documentation that says that in order to make
> > >> tests pass you have to enable the flag
> > >>
> > >> That flag is deprecated and enabled by default in modern JREs, so I
> > >> would go for 2) or 3)
> > >>
> > >> I guess that on  ASF Jenkins if the JDK8 we are using has the flag
> > turned
> > >> on
> > >>
> > >> Enrico
> > >>
> > >> Il giorno sab 27 apr 2019 alle ore 17:48 Andor Molnar
> > >> <andor@apache.org> ha scritto:
> > >>>
> > >>> I’m running the tests fine without setting the policy to unlimited:
> > >>>
> > >>> java version "1.8.0_161"
> > >>> Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
> > >>> Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)
> > >>>
> > >>> Have you tried to run it with a more recent version of Java?
> > >>>
> > >>> Andor
> > >>>
> > >>>
> > >>>
> > >>>> On 2019. Apr 27., at 17:33, Andor Molnar <andor@apache.org>
wrote:
> > >>>>
> > >>>> Good catch, thanks Flavio for reporting this. We need to double
> check
> > >> the tests with Ilya I believe.
> > >>>>
> > >>>> Having tests failure means that you were actually able to _build_
> > >> ZooKeeper successfully without changing the crypto policy setting.
> Have
> > you
> > >> tried to start an ensemble with Quorum TLS by any chance? That would
> add
> > >> some more color to this issue.
> > >>>>
> > >>>> This might be just a testing issue.
> > >>>>
> > >>>> Regards,
> > >>>> Andor
> > >>>>
> > >>>>
> > >>>>
> > >>>>> On 2019. Apr 27., at 16:09, Flavio Junqueira <fpj@apache.org>
> wrote:
> > >>>>>
> > >>>>> Hi Enrico,
> > >>>>>
> > >>>>> Here is the info you are requesting:
> > >>>>>
> > >>>>> *Java version*
> > >>>>>
> > >>>>> $ java -version
> > >>>>> java version "1.8.0_152"
> > >>>>> Java(TM) SE Runtime Environment (build 1.8.0_152-b16)
> > >>>>> Java HotSpot(TM) 64-Bit Server VM (build 25.152-b16, mixed
mode)
> > >>>>>
> > >>>>> *Test case errors*
> > >>>>>
> > >>>>> I won’t post all of them, I get a good number of errors:
> > >>>>>
> > >>>>> ================================
> > >>>>> [ERROR] Tests run: 64, Failures: 0, Errors: 16, Skipped: 0,
Time
> > >> elapsed: 9.21 s <<< FAILURE! - in
> > org.apache.zookeeper.util.PemReaderTest
> > >>>>> [ERROR]
> > >>
> >
> testLoadCertificateFromKeyStore[1](org.apache.zookeeper.util.PemReaderTest)
> > >> Time elapsed: 1.593 s  <<< ERROR!
> > >>>>> java.io.IOException:
> > >> org.bouncycastle.operator.OperatorCreationException: Illegal key size
> or
> > >> default parameters
> > >>>>>     at
> > >>
> >
> org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(PemReaderTest.java:125)
> > >>>>> Caused by: org.bouncycastle.operator.OperatorCreationException:
> > >> Illegal key size or default parameters
> > >>>>>     at
> > >>
> >
> org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(PemReaderTest.java:125)
> > >>>>> Caused by: java.security.InvalidKeyException: Illegal key size
or
> > >> default parameters
> > >>>>>     at
> > >>
> >
> org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(PemReaderTest.java:125)
> > >>>>>
> > >>>>> [ERROR]
> > >>
> >
> testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword[1](org.apache.zookeeper.util.PemReaderTest)
> > >> Time elapsed: 0.004 s  <<< ERROR!
> > >>>>> java.lang.Exception: Unexpected exception,
> > >> expected<java.security.GeneralSecurityException> but
> > >> was<java.io.IOException>
> > >>>>>     at
> > >>
> >
> org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword(PemReaderTest.java:93)
> > >>>>> Caused by: org.bouncycastle.operator.OperatorCreationException:
> > >> Illegal key size or default parameters
> > >>>>>     at
> > >>
> >
> org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword(PemReaderTest.java:93)
> > >>>>> Caused by: java.security.InvalidKeyException: Illegal key size
or
> > >> default parameters
> > >>>>>     at
> > >>
> >
> org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword(PemReaderTest.java:93)
> > >>>>> ...
> > >>>>> ================================
> > >>>>>
> > >>>>>
> > >>>>> *Crypto policy*
> > >>>>> If I uncomment this configuration option:
> > >>>>>
> > >>>>> # Please see the JCA documentation for additional information
on
> > these
> > >>>>> # files and formats.
> > >>>>> # crypto.policy=unlimited
> > >>>>>
> > >>>>> in:
> > >>>>>
> > >>>>> $JAVA_HOME/jre/lib/security/java.security
> > >>>>>
> > >>>>> then it all works and I get no error at all. This option controls
> > >> cryptographic strengths according to the documentation, and is present
> > >> because of crypto regulations in different countries.
> > >>>>>
> > >>>>> Thanks,
> > >>>>> -Flavio
> > >>>>>
> > >>>>>> On 27 Apr 2019, at 15:52, Enrico Olivelli <eolivelli@gmail.com>
> > >> wrote:
> > >>>>>>
> > >>>>>> Il sab 27 apr 2019, 14:18 Flavio Junqueira <fpj@apache.org>
ha
> > >> scritto:
> > >>>>>>
> > >>>>>>> I have a clarification question about the RC. To build
the RC, I
> > >> had to
> > >>>>>>> enable crypto.policy unlimited in the jre (I'm using
build
> > >> 1.8.0_152-b16).
> > >>>>>>
> > >>>>>>
> > >>>>>> Flavio
> > >>>>>> What do you mean with 'build' ?
> > >>>>>> Make tests pass?
> > >>>>>> AFAIK we are not using tweaked jdks in CI builds, so in
theory
> there
> > >> is no
> > >>>>>> need.
> > >>>>>>
> > >>>>>> Can you please share your error?
> > >>>>>>
> > >>>>>> Enrico
> > >>>>>>
> > >>>>>>
> > >>>>>> I'm wondering if this is going to be an issue for some
users as
> this
> > >> option
> > >>>>>>> is related to import/export regulation. Has anyone
looked into it
> > >> and could
> > >>>>>>> clarify it to me, please?
> > >>>>>>>
> > >>>>>>> Thanks,
> > >>>>>>> -Flavio
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>> On 25 Apr 2019, at 15:10, Andor Molnar <andor@apache.org>
> wrote:
> > >>>>>>>>
> > >>>>>>>> This is the first stable release of 3.5 branch:
3.5.5. It
> resolves
> > >> 117
> > >>>>>>> issues, including Maven migration, Quorum TLS, TTL
nodes and lots
> > >> of other
> > >>>>>>> performance and stability improvements.
> > >>>>>>>>
> > >>>>>>>> The full release notes is available at:
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801&version=12343268
> > >>>>>>>>
> > >>>>>>>> *** Please download, test and vote by May 3rd 2019,
23:59 UTC+0.
> > >> ***
> > >>>>>>>>
> > >>>>>>>> Source files:
> > >>>>>>>>
> > >> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.5.5-rc5/
> > >>>>>>>>
> > >>>>>>>> Maven staging repos:
> > >>>>>>>>
> > >>>>>>>
> > >>
> >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/parent/3.5.5/
> > >>>>>>>>
> > >>>>>>>
> > >>
> >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper-jute/3.5.5/
> > >>>>>>>>
> > >>>>>>>
> > >>
> >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.5/
> > >>>>>>>>
> > >>>>>>>> The release candidate tag in git to be voted upon:
> > >> release-3.5.5-rc5
> > >>>>>>>>
> > >>>>>>>> ZooKeeper's KEYS file containing PGP keys we use
to sign the
> > >> release:
> > >>>>>>>> http://www.apache.org/dist/zookeeper/KEYS
> > >>>>>>>>
> > >>>>>>>> Should we release this candidate?
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message