From dev-return-80570-archive-asf-public=cust-asf.ponee.io@zookeeper.apache.org Mon Apr 29 14:54:01 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id D8D8A18061A for ; Mon, 29 Apr 2019 16:54:00 +0200 (CEST) Received: (qmail 82619 invoked by uid 500); 29 Apr 2019 14:53:59 -0000 Mailing-List: contact dev-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@zookeeper.apache.org Delivered-To: mailing list dev@zookeeper.apache.org Received: (qmail 82598 invoked by uid 99); 29 Apr 2019 14:53:59 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 29 Apr 2019 14:53:58 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id E6573C2283 for ; Mon, 29 Apr 2019 14:53:57 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.001 X-Spam-Level: X-Spam-Status: No, score=-0.001 tagged_above=-999 required=6.31 tests=[HEADER_FROM_DIFFERENT_DOMAINS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 2w3zuLlA84R7 for ; Mon, 29 Apr 2019 14:53:55 +0000 (UTC) Received: from mail-wm1-f67.google.com (mail-wm1-f67.google.com [209.85.128.67]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id A7D485F118 for ; Mon, 29 Apr 2019 14:53:55 +0000 (UTC) Received: by mail-wm1-f67.google.com with SMTP id h18so16267174wml.1 for ; Mon, 29 Apr 2019 07:53:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:mime-version :subject:date:references:to:in-reply-to:message-id; bh=IYsYLWLmrKIlgIGKzE82ezxBV1yynsgsg1iI1VVhxKA=; b=HaULh6mxRO9mBTnQkBG0+HalfMqk3DGZCqSj66adj5U/lZu66PoZWvj2nxXRkxidOn BVwHlW7sZi66C+/xf134f7HL95XCVRtKm10VXq8q84cCBvGHpBNC2K81Hkz8Rnk4CyPb V25RoT0+ilkofZvvun58Zhb3z3zLNzbkKqf4SG5nKkxrxJf7wGRHwEO7vxUsk5xBD13N y+jVrpBjwW1iFZ4hhvbel/XipjZRCmPIUMscIgO163tmoo+i76CHuFj7HldXkJlbX/6p DQTMzOOam41Tt690aDJWgCVx77tvJaiaJcnmLbX0JmzTLuyTLrLoBmalrAkt3aIGa5A2 iu8Q== X-Gm-Message-State: APjAAAVjI023DazQ6/VfaebbcPZDo2DgIjfv+//uwdJ5Edynyptvmip2 Rj/xdi3j9WtOn9NRbtcGOW2yBnU8 X-Google-Smtp-Source: APXvYqzXIwddLX9DPt/mOGDdp3MGKBkEllguV3VYRlDfPC6ARuddkk1FAiPI6p26ThQ+4bHVdkEnYg== X-Received: by 2002:a7b:c115:: with SMTP id w21mr17505347wmi.55.1556549634397; Mon, 29 Apr 2019 07:53:54 -0700 (PDT) Received: from [192.168.1.36] (138.red-79-159-27.dynamicip.rima-tde.net. [79.159.27.138]) by smtp.gmail.com with ESMTPSA id l5sm21809090wrb.51.2019.04.29.07.53.53 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 29 Apr 2019 07:53:53 -0700 (PDT) From: Flavio Junqueira Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.0 \(3445.100.39\)) Subject: Re: Crypto Policy (was: Re: [VOTE] Apache ZooKeeper release 3.5.5 candidate 5) Date: Mon, 29 Apr 2019 16:53:52 +0200 References: <6C53CD6A-E7E6-42F1-91C7-D16517381F75@apache.org> <2F761131-C8CC-4C4D-A8F3-B97C369982E7@apache.org> To: dev@zookeeper.apache.org In-Reply-To: Message-Id: <526FB840-157C-43B9-9754-9A0567A81E19@apache.org> X-Mailer: Apple Mail (2.3445.100.39) I'm also +1 for adding a comment to the release notes (thanks for the = suggestion, Ted). Updating the readme makes sense, but the release notes = will be the main source to indicate that we require a specific or later = version of Java from that particular release. My preference would be to = update the release notes. As for running TLS on a single node, have you been able to do it? I = haven't had a chance to look further into it throughout my day, so if = anyone has successfully done it and can share some instructions, it = would help me. Otherwise, I'll keep investigating once I have a chance. = To be specific, I created the keystore, certificate and truststore files = according to instructions, but the instructions assume that the aliases = are different when it comes to populating the truststore. At that point, = I had to get creative and I have tried a couple of options that didn't = work. Either way, I think that being able to run locally and documenting = is desirable, although not a blocker. If I can get it right, then I can = write a gist describing it that we can use as a reference until we = properly document it. -Flavio > On 29 Apr 2019, at 15:42, Andor Molnar wrote: >=20 > Thanks Flavio for the investigation. I=E2=80=99ll update the README = file to include instructions on supported Java 8 versions. > I=E2=80=99m wondering if I have to update the admin docs based on your = problems running TLS quorum on a single machine. >=20 > Andor >=20 >=20 >=20 >> On 2019. Apr 29., at 15:06, Enrico Olivelli = wrote: >>=20 >> Il lun 29 apr 2019, 13:44 Ted Dunning ha = scritto: >>=20 >>> Other changes in u211+ substantially improve how Java 8 applications = behave >>> in containers. I am seeing this more and more with customers. >>>=20 >>> Combined with the crypto issues, it might be worth a release note >>> suggesting that if you are going to compile with Java 1.8, you = should use a >>> recent release at u211 (u212?) Or above. >>>=20 >>=20 >> +1 for a note on release docs >>=20 >>=20 >> Enrico >>=20 >>=20 >>=20 >>=20 >>> On Mon, Apr 29, 2019, 11:43 AM Flavio Junqueira = wrote: >>>=20 >>>> I did a bit more research and it turns out that the crypto.policy = option >>>> was introduced u151: >>>>=20 >>>>=20 >>> = https://www.oracle.com/technetwork/java/javase/8u151-relnotes-3850493.html= >>>> < >>>>=20 >>> = https://www.oracle.com/technetwork/java/javase/8u151-relnotes-3850493.html= >>>>>=20 >>>>=20 >>>> And started being defined by default with the "unlimited" option in = u161: >>>>=20 >>>>=20 >>> = https://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html= >>>> < >>>>=20 >>> = https://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html= >>>>>=20 >>>>=20 >>>> I have installed a more recent version, 1.8.0_211, and it builds = fine >>> (all >>>> tests pass consistently for me). >>>>=20 >>>>=20 >>>> I'm now trying to start an ensemble with ssl enabled locally, but = it is >>>> failing for me. It looks like the instructions in the admin doc = assumes >>>> different hosts. I need to look more closely into it to determine = what >>> not >>>> is that I'm doing wrong, but in any case, the instructions do not = make it >>>> very clear whether one can run locally. >>>>=20 >>>> -Flavio >>>>=20 >>>>> On 27 Apr 2019, at 19:28, Patrick Hunt wrote: >>>>>=20 >>>>> Odd. I had done my testing on jdk11/macos which is fine. >>>>>=20 >>>>> I just tried jdk8 and 3 times in a row it's failing with: >>>>> [ERROR] = SaslAuthTest.testZKOperationsAfterClientSaslAuthFailure:176 =C2=BB >>>>> Timeout Failed t... >>>>>=20 >>>>> I don't see the error Flavio is seeing. I have never installed = special >>>>> crypto libraries, etc... just vanilla jdk. >>>>>=20 >>>>> =E2=8C=82102% [phunt:~/Downloads/z/apache-zookeeper-3.5.5] 3s $ = mvn --version >>>>> Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; >>>>> 2019-04-04T12:00:29-07:00) >>>>> Maven home: /usr/local/Cellar/maven/3.6.1/libexec >>>>> Java version: 1.8.0_201, vendor: Oracle Corporation, runtime: >>>>> = /Library/Java/JavaVirtualMachines/jdk1.8.0_201.jdk/Contents/Home/jre >>>>> Default locale: en_US, platform encoding: UTF-8 >>>>> OS name: "mac os x", version: "10.14.4", arch: "x86_64", family: = "mac" >>>>>=20 >>>>>=20 >>>>> 2019-04-27 10:11:51,635 [myid:] - INFO >>>>> [NIOWorkerThread-6:SaslServerCallbackHandler@136] - Setting >>>> authorizedID: >>>>> super >>>>> 2019-04-27 10:11:51,636 [myid:] - INFO >>>>> [NIOWorkerThread-6:ZooKeeperServer@1170] - adding SASL = authorization >>> for >>>>> authorizationID: super >>>>> 2019-04-27 10:11:51,813 [myid:] - INFO >>>>> [SessionTracker:SessionTrackerImpl@163] - SessionTrackerImpl = exited >>>> loop! >>>>> 2019-04-27 10:12:21,596 [myid:] - INFO >>>>> [main:JUnit4ZKTestRunner$LoggedInvokeMethod@98] - TEST METHOD = FAILED >>>>> testZKOperationsAfterClientSaslAuthFailure >>>>> java.util.concurrent.TimeoutException: Failed to connect to = ZooKeeper >>>>> server. >>>>> at >>>>>=20 >>>>=20 >>> = org.apache.zookeeper.test.ClientBase$CountdownWatcher.waitForConnected(Cli= entBase.java:150) >>>>> at >>>>>=20 >>>>=20 >>> = org.apache.zookeeper.SaslAuthTest.testZKOperationsAfterClientSaslAuthFailu= re(SaslAuthTest.java:176) >>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>> at >>>>>=20 >>>>=20 >>> = sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:= 62) >>>>> at >>>>>=20 >>>>=20 >>> = sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm= pl.java:43) >>>>> at java.lang.reflect.Method.invoke(Method.java:498) >>>>> at >>>>>=20 >>>>=20 >>> = org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMetho= d.java:50) >>>>> at >>>>>=20 >>>>=20 >>> = org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable= .java:12) >>>>>=20 >>>>>=20 >>>>> Patrick >>>>>=20 >>>>>=20 >>>>> On Sat, Apr 27, 2019 at 8:54 AM Enrico Olivelli = >>>> wrote: >>>>>=20 >>>>>> On my local tests I usually don't get the error because I am = using >>>>>> jdk11 and unlimited strength cryptography is enabled by default >>>>>>=20 >>>>>>=20 >>>>=20 >>> = https://www.oracle.com/technetwork/java/javase/documentation/jdk11-readme-= 5097204.html#jce >>>>>>=20 >>>>>> In production it depends on the configuration of SSL, if you = require >>>>>> strong ciphers/big keys you will have problems, but the server = won't >>>>>> start so you will find soon the problem. >>>>>> I think this is not a real issue (for production I mean). >>>>>> I see these ways: >>>>>> 1) adapt the tests in order to make default jdk8 happy >>>>>> 2) tweak the tests enabling "unlimited strenght cryptography" = using >>>>>> reflection >>>>>> 3) just write a line in documentation that says that in order to = make >>>>>> tests pass you have to enable the flag >>>>>>=20 >>>>>> That flag is deprecated and enabled by default in modern JREs, so = I >>>>>> would go for 2) or 3) >>>>>>=20 >>>>>> I guess that on ASF Jenkins if the JDK8 we are using has the = flag >>>> turned >>>>>> on >>>>>>=20 >>>>>> Enrico >>>>>>=20 >>>>>> Il giorno sab 27 apr 2019 alle ore 17:48 Andor Molnar >>>>>> ha scritto: >>>>>>>=20 >>>>>>> I=E2=80=99m running the tests fine without setting the policy to = unlimited: >>>>>>>=20 >>>>>>> java version "1.8.0_161" >>>>>>> Java(TM) SE Runtime Environment (build 1.8.0_161-b12) >>>>>>> Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode) >>>>>>>=20 >>>>>>> Have you tried to run it with a more recent version of Java? >>>>>>>=20 >>>>>>> Andor >>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>>> On 2019. Apr 27., at 17:33, Andor Molnar = wrote: >>>>>>>>=20 >>>>>>>> Good catch, thanks Flavio for reporting this. We need to double >>> check >>>>>> the tests with Ilya I believe. >>>>>>>>=20 >>>>>>>> Having tests failure means that you were actually able to = _build_ >>>>>> ZooKeeper successfully without changing the crypto policy = setting. >>> Have >>>> you >>>>>> tried to start an ensemble with Quorum TLS by any chance? That = would >>> add >>>>>> some more color to this issue. >>>>>>>>=20 >>>>>>>> This might be just a testing issue. >>>>>>>>=20 >>>>>>>> Regards, >>>>>>>> Andor >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>> On 2019. Apr 27., at 16:09, Flavio Junqueira >>> wrote: >>>>>>>>>=20 >>>>>>>>> Hi Enrico, >>>>>>>>>=20 >>>>>>>>> Here is the info you are requesting: >>>>>>>>>=20 >>>>>>>>> *Java version* >>>>>>>>>=20 >>>>>>>>> $ java -version >>>>>>>>> java version "1.8.0_152" >>>>>>>>> Java(TM) SE Runtime Environment (build 1.8.0_152-b16) >>>>>>>>> Java HotSpot(TM) 64-Bit Server VM (build 25.152-b16, mixed = mode) >>>>>>>>>=20 >>>>>>>>> *Test case errors* >>>>>>>>>=20 >>>>>>>>> I won=E2=80=99t post all of them, I get a good number of = errors: >>>>>>>>>=20 >>>>>>>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>>>>>>>> [ERROR] Tests run: 64, Failures: 0, Errors: 16, Skipped: 0, = Time >>>>>> elapsed: 9.21 s <<< FAILURE! - in >>>> org.apache.zookeeper.util.PemReaderTest >>>>>>>>> [ERROR] >>>>>>=20 >>>>=20 >>> = testLoadCertificateFromKeyStore[1](org.apache.zookeeper.util.PemReaderTest= ) >>>>>> Time elapsed: 1.593 s <<< ERROR! >>>>>>>>> java.io.IOException: >>>>>> org.bouncycastle.operator.OperatorCreationException: Illegal key = size >>> or >>>>>> default parameters >>>>>>>>> at >>>>>>=20 >>>>=20 >>> = org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(Pe= mReaderTest.java:125) >>>>>>>>> Caused by: = org.bouncycastle.operator.OperatorCreationException: >>>>>> Illegal key size or default parameters >>>>>>>>> at >>>>>>=20 >>>>=20 >>> = org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(Pe= mReaderTest.java:125) >>>>>>>>> Caused by: java.security.InvalidKeyException: Illegal key size = or >>>>>> default parameters >>>>>>>>> at >>>>>>=20 >>>>=20 >>> = org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(Pe= mReaderTest.java:125) >>>>>>>>>=20 >>>>>>>>> [ERROR] >>>>>>=20 >>>>=20 >>> = testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword[1](org.apache.zoo= keeper.util.PemReaderTest) >>>>>> Time elapsed: 0.004 s <<< ERROR! >>>>>>>>> java.lang.Exception: Unexpected exception, >>>>>> expected but >>>>>> was >>>>>>>>> at >>>>>>=20 >>>>=20 >>> = org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKey= StoreWithWrongPassword(PemReaderTest.java:93) >>>>>>>>> Caused by: = org.bouncycastle.operator.OperatorCreationException: >>>>>> Illegal key size or default parameters >>>>>>>>> at >>>>>>=20 >>>>=20 >>> = org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKey= StoreWithWrongPassword(PemReaderTest.java:93) >>>>>>>>> Caused by: java.security.InvalidKeyException: Illegal key size = or >>>>>> default parameters >>>>>>>>> at >>>>>>=20 >>>>=20 >>> = org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKey= StoreWithWrongPassword(PemReaderTest.java:93) >>>>>>>>> ... >>>>>>>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> *Crypto policy* >>>>>>>>> If I uncomment this configuration option: >>>>>>>>>=20 >>>>>>>>> # Please see the JCA documentation for additional information = on >>>> these >>>>>>>>> # files and formats. >>>>>>>>> # crypto.policy=3Dunlimited >>>>>>>>>=20 >>>>>>>>> in: >>>>>>>>>=20 >>>>>>>>> $JAVA_HOME/jre/lib/security/java.security >>>>>>>>>=20 >>>>>>>>> then it all works and I get no error at all. This option = controls >>>>>> cryptographic strengths according to the documentation, and is = present >>>>>> because of crypto regulations in different countries. >>>>>>>>>=20 >>>>>>>>> Thanks, >>>>>>>>> -Flavio >>>>>>>>>=20 >>>>>>>>>> On 27 Apr 2019, at 15:52, Enrico Olivelli = >>>>>> wrote: >>>>>>>>>>=20 >>>>>>>>>> Il sab 27 apr 2019, 14:18 Flavio Junqueira = ha >>>>>> scritto: >>>>>>>>>>=20 >>>>>>>>>>> I have a clarification question about the RC. To build the = RC, I >>>>>> had to >>>>>>>>>>> enable crypto.policy unlimited in the jre (I'm using build >>>>>> 1.8.0_152-b16). >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> Flavio >>>>>>>>>> What do you mean with 'build' ? >>>>>>>>>> Make tests pass? >>>>>>>>>> AFAIK we are not using tweaked jdks in CI builds, so in = theory >>> there >>>>>> is no >>>>>>>>>> need. >>>>>>>>>>=20 >>>>>>>>>> Can you please share your error? >>>>>>>>>>=20 >>>>>>>>>> Enrico >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> I'm wondering if this is going to be an issue for some users = as >>> this >>>>>> option >>>>>>>>>>> is related to import/export regulation. Has anyone looked = into it >>>>>> and could >>>>>>>>>>> clarify it to me, please? >>>>>>>>>>>=20 >>>>>>>>>>> Thanks, >>>>>>>>>>> -Flavio >>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>>>> On 25 Apr 2019, at 15:10, Andor Molnar >>> wrote: >>>>>>>>>>>>=20 >>>>>>>>>>>> This is the first stable release of 3.5 branch: 3.5.5. It >>> resolves >>>>>> 117 >>>>>>>>>>> issues, including Maven migration, Quorum TLS, TTL nodes and = lots >>>>>> of other >>>>>>>>>>> performance and stability improvements. >>>>>>>>>>>>=20 >>>>>>>>>>>> The full release notes is available at: >>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>=20 >>>>=20 >>> = https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=3D1231080= 1&version=3D12343268 >>>>>>>>>>>>=20 >>>>>>>>>>>> *** Please download, test and vote by May 3rd 2019, 23:59 = UTC+0. >>>>>> *** >>>>>>>>>>>>=20 >>>>>>>>>>>> Source files: >>>>>>>>>>>>=20 >>>>>> = https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.5.5-rc5/ >>>>>>>>>>>>=20 >>>>>>>>>>>> Maven staging repos: >>>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>=20 >>>>=20 >>> = https://repository.apache.org/content/groups/staging/org/apache/zookeeper/= parent/3.5.5/ >>>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>=20 >>>>=20 >>> = https://repository.apache.org/content/groups/staging/org/apache/zookeeper/= zookeeper-jute/3.5.5/ >>>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>=20 >>>>=20 >>> = https://repository.apache.org/content/groups/staging/org/apache/zookeeper/= zookeeper/3.5.5/ >>>>>>>>>>>>=20 >>>>>>>>>>>> The release candidate tag in git to be voted upon: >>>>>> release-3.5.5-rc5 >>>>>>>>>>>>=20 >>>>>>>>>>>> ZooKeeper's KEYS file containing PGP keys we use to sign = the >>>>>> release: >>>>>>>>>>>> http://www.apache.org/dist/zookeeper/KEYS >>>>>>>>>>>>=20 >>>>>>>>>>>> Should we release this candidate? >>>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>=20 >>>>>>>=20 >>>>>>=20 >>>>=20 >>>>=20 >>>=20 >=20