From dev-return-83095-archive-asf-public=cust-asf.ponee.io@zookeeper.apache.org Thu Oct 3 18:34:59 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 8400218065B for ; Thu, 3 Oct 2019 20:34:59 +0200 (CEST) Received: (qmail 52051 invoked by uid 500); 3 Oct 2019 18:34:58 -0000 Mailing-List: contact dev-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@zookeeper.apache.org Delivered-To: mailing list dev@zookeeper.apache.org Received: (qmail 52040 invoked by uid 99); 3 Oct 2019 18:34:58 -0000 Received: from Unknown (HELO mailrelay1-lw-us.apache.org) (10.10.3.42) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Oct 2019 18:34:58 +0000 Received: from dph-mint (51B61F18.dsl.pool.telekom.hu [81.182.31.24]) by mailrelay1-lw-us.apache.org (ASF Mail Server at mailrelay1-lw-us.apache.org) with ESMTPSA id D2CA46222 for ; Thu, 3 Oct 2019 18:34:57 +0000 (UTC) Message-ID: <5134985b8edfda78fe3aafd0a3790a359c354929.camel@apache.org> Subject: Re: [VOTE] Apache ZooKeeper release 3.5.6 candidate 2 From: Andor Molnar To: dev@zookeeper.apache.org Date: Thu, 03 Oct 2019 20:34:56 +0200 In-Reply-To: References: <4693A138-7017-4B45-A39D-A9816B412C1F@apache.org> <484431A6-456C-4159-9FE4-CF49E9C7C160@apache.org> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.30.5-1.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Looks like we only need some refactoring on the testing side: testRaceBetweenSyncFlushAndZKShutdown() uses SimpleZooKeeperServer class which is based on Netty and needs to be refactored to use NIO instead. Otherwise looks like a quite straightforward change. +1 for removing from the codebase and release 3.4.15 without Netty. Andor -----Original Message----- From: Patrick Hunt Reply-To: dev@zookeeper.apache.org To: DevZooKeeper Subject: Re: [VOTE] Apache ZooKeeper release 3.5.6 candidate 2 Date: Thu, 3 Oct 2019 07:36:24 -0700 On Wed, Oct 2, 2019 at 9:59 PM Brian Nixon < brian.nixon.cs@gmail.com > wrote: > NIO is still the default server factory so I'm guessing many users of > 3.4 > simply aren't configuring Netty. And our recommendation for users who > want > Netty could be to upgrade to a 3.5 release as that should be better > in > every way for them. > > Is there a principle determining the difference between leaving the > code > available in 3.4 with a warning attached and removing the code > entirely so > that they would have to independently modify and package in order to > use > the feature? > > Primarily/historically what i mentioned - we don't introduce major features/changes (esp non-b/w compat) in fix releases. Patrick > On Wed, Oct 2, 2019 at 8:48 AM Patrick Hunt < > phunt@apache.org > > wrote: > > > On Wed, Oct 2, 2019 at 1:49 AM Andor Molnar < > > andor@apache.org > > > wrote: > > > > > Hi Pat, > > > > > > Would you please clarify what do you mean “dropping netty support > > > from > > > 3.4”? > > > > > > > > > > My simplistic thought was just that. Ship new versions of 3.4 that > > remove > > support for netty. That could mean turning if off by default (not > > sure > > how > > much work that would be) or just purging the netty code from the > > codebase > > entirely. (3.4). It would be an exception to our "don't break b/w > > compact > > in fix releases" policy, but this is an extreme case imo. We have > > no > > intention of supporting netty in 3.4 going forward as evidenced by > > the > > fact > > that the netty version is locked to netty 3 (long out of support by > > netty > > as they are no longer backporting fixes) and we have no intention > > of > > updating to the new version of netty on 3.4. Maybe this CVE don't > > affect > > us, but at some point it will. Users have the option to move to a > > stable, > > b/w compat, 3.5. release. Not optimal I agree. > > > > > > > Does that mean we won’t submit security patches from now on, but > > > keep > > the > > > Netty classes (NettyServerCnxnFactory and ClientCnxnSocketNetty) > > > > available > > > OR remove these classes from the codebase? > > > > > > The latter means we’ll drop client SSL feature too. > > > > > > > > > > Say there is a new CVE on netty and it's not backported to netty3, > > what > > would we do in that case. I guess we could wait/kick the can down > > the > > road > > till we really hit that. For the moment just say that it doesn't > > affect > > us > > as you researched and add to 3.4 exceptions. > > > > This is just my suggestion/option rather than a recommendation, > > open to > > other ideas. ;-) > > > > Patrick > > > > > > > Andor > > > > > > > > > > > > > On 2019. Oct 2., at 2:27, Michael Han < > > > > hanm@apache.org > > > > > wrote: > > > > > > > > > > How about officially dropping netty support from 3.4 and > > > > > > asking > > > > people > > > > to move to the new version > > > > +1. This sounds a good opportunity to deprecate 3.4 branch. > > > > > > > > On Tue, Oct 1, 2019 at 8:00 AM Enrico Olivelli < > > > > eolivelli@gmail.com > > > > > > > > > > > wrote: > > > > > Il giorno mar 1 ott 2019 alle ore 16:15 Patrick Hunt < > > > > phunt@apache.org > > > > > > ha > > > > > scritto: > > > > > > > > > > > Another option/solution: How about officially dropping > > > > > > netty > > support > > > from > > > > > > 3.4 and asking people to move to the new version (3.5 > > > > > > stable or > > > > later)? > > > > > > > > > > Sounds good > > > > > > > > > > Enrico > > > > > > > > > > > > > > > > Patrick > > > > > > > > > > > > On Tue, Oct 1, 2019 at 4:22 AM Andor Molnar < > > > > > > andor@apache.org > > > > > > > > > > > wrote: > > > > > > > I agree with 3.4 should not be refactored in any way even > > > > > > > for a > > > > > > > > > > security > > > > > > > fix. > > > > > > > > > > > > > > What's wrong with the "alpha story"? > > > > > > > > > > > > > > I think releasing in an early stage with "-alpha", "- > > > > > > > beta" > > modifiers > > > is > > > > > > > not a bad thing alone, as long as it doesn't take years > > > > > > > to get to > > > > the > > > > > > > stable release. > > > > > > > > > > > > > > Andor > > > > > > > > > > > > > > > > > > > > > On Tue, 1 Oct 2019, Enrico Olivelli wrote: > > > > > > > > > > > > > > > Date: Tue, 1 Oct 2019 10:54:24 +0200 > > > > > > > > From: Enrico Olivelli < > > > > > > > > eolivelli@gmail.com > > > > > > > > > > > > > > > > > Reply-To: > > > > > > > > dev@zookeeper.apache.org > > > > > > > > > > > > > > > > To: > > > > > > > > dev@zookeeper.apache.org > > > > > > > > > > > > > > > > Subject: Re: [VOTE] Apache ZooKeeper release 3.5.6 > > > > > > > > candidate 2 > > > > > > > > > > > > > > > > Il mar 1 ott 2019, 10:38 Andor Molnar < > > > > > > > > andor@apache.org > > > > > > > > > ha > > > > scritto: > > > > > > > > > Backporting Netty 4 would be a huge, cumbersome task, > > > > > > > > > I hope we > > > > > > > > > > don’t > > > > > > > have > > > > > > > > > to do it. > > > > > > > > > > > > > > > > > > > > > > > > > Yes, 3.4 is mature and stable and closed for refactors. > > > > > > > > > > > > > > > > > > > > > > > > > However I had a quick look at the details of this CVE > > > > > > > > > and it > > seems > > > > > to > > > > > > me > > > > > > > > > that it only affects the HTTP codec: > > > > > > > > > > > > > > > > > > > > https://github.com/netty/netty/commit/39cafcb05c99f2aa9fce7e6597664c9ed6a63a95 > > > > > > > > > > Can’t we just say 3.4.14 is not affected? > > > > > > > > > We’re not running HTTP server inside ZooKeeper. > > > > > > > > > > > > > > > > > > Otherwise we might be able to release 3.6.0-alpha1 > > > > > > > > > now, put a > > date > > > > > for > > > > > > > 3.4 > > > > > > > > > EOL and highlight on the webpage that this > > > > > > > > > > > > > > > > > > > > > > > > > Please do not start an 'alpha' story like for 3.5.... > > > > > > > > > > > > > > > > CVE probably won’t be resolved on that branch, please > > > > > > > > upgrade to > > > > 3.5. > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > > > > > Enrico > > > > > > > > > > > > > > > > > As a third option we could ask Norman to kindly fix > > > > > > > > > 3.10.6.Final > > > > as > > > > > > > well… > > > > > > > > > or submit a PR ourselves, it doesn’t seem to me a big > > > > > > > > > deal. > > > > > > > > > > > > > > > > > > > > > > > > > Not so useful > > > > > > > > > > > > > > > > > What do you think? > > > > > > > > > > > > > > > > > > Andor > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 2019. Oct 1., at 2:00, Patrick Hunt < > > > > > > > > > > phunt@apache.org > > > > > > > > > > > > > wrote: > > > > > > > > > > I pushed patches for 3.5 and trunk and the tests > > > > > > > > > > passed on my > > > > mac. > > > > > > > > > However > > > > > > > > > > 3.4 is using netty 3.10.6.Final and as such it's > > > > > > > > > > not a simple > > > > > > > > > > > > upgrade. > > > > > > > > > > (there are no fixes against 3.10 for this CVE, at > > > > > > > > > > least not so > > > > far) > > > > > > Not > > > > > > > > > > sure what we want to do about this... someone would > > > > > > > > > > need to > > > > > > > > > > backport > > > > > > > the > > > > > > > > > > netty 4.1 changes into 3.4 afaict. > > > > > > > > > > > > > > > > > > > > Patrick > > > > > > > > > > > > > > > > > > > > On Mon, Sep 30, 2019 at 1:08 PM Patrick Hunt < > > > > > > > > > > phunt@apache.org > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > I'll work on it today. > > > > > > > > > > > > > > > > > > > > > > Patrick > > > > > > > > > > > > > > > > > > > > > > On Mon, Sep 30, 2019 at 11:59 AM Enrico Olivelli > > > > > > > > > > > < > > > > > > > > > > > > eolivelli@gmail.com > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Okay > > > > > > > > > > > > > > > > > > > > > > > > I am cancelling the release. > > > > > > > > > > > > > > > > > > > > > > > > I have a problem with my box, I can't work on > > > > > > > > > > > > netty upgrade. > > > > > > > > > > > > > > > > > > > > > > > > Any volounteer? > > > > > > > > > > > > > > > > > > > > > > > > Enrico > > > > > > > > > > > > > > > > > > > > > > > > Il lun 30 set 2019, 20:32 Andor Molnar < > > > > > > > > > > > > andor@apache.org > > > > > > > > > > > > > ha > > > > > > > > > > > > > > scritto: > > > > > > > > > > > > > The good news is: we need to release 3.4.15 > > > > > > > > > > > > > too. :) > > > > > > > > > > > > > > > > > > > > > > > > > > Andor > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 2019. Sep 30., at 20:26, Patrick Hunt < > > > > > > > > > > > > > > phunt@apache.org > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > created: > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-3563 > > > > > > > > > > > > > > > > On Mon, Sep 30, 2019 at 11:20 AM Patrick > > > > > > > > > > > > > > Hunt < > > > > > > > > > > phunt@apache.org > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > -1 - when I run dependency check on the > > > > > > > > > > > > > > > release candidate > > > > > > > > > > > > artifact > > > > > > > > > > > > it's > > > > > > > > > > > > > > > failing with: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [ERROR] netty-transport-4.1.29.Final.jar: > > > > > > > > > > > > > > > CVE-2019-16869 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I ran this on trunk and it's passing, as > > > > > > > > > > > > > > > such it must be > > an > > > > > > issue > > > > > > > > > > > > with > > > > > > > > > > > > > the > > > > > > > > > > > > > > > the 3.5.6 netty version specifically. > > > > > > > > > > > > > > > It's listed as a > > high, > > > > > we > > > > > > > > > > > > should > > > > > > > > > > > > > > > patch this as well before releasing. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Patrick > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Sep 29, 2019 at 7:29 AM Enrico > > > > > > > > > > > > > > > Olivelli < > > > > > > > > > > > > > > > > > > eolivelli@gmail.com > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This is a bugfix release candidate for > > > > > > > > > > > > > > > > 3.5.6. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It fixes 28 issues, including upgrade > > > > > > > > > > > > > > > > of third party > > > > > > > > > > libraries, > > > > > > > > > > > > > > > > TTL Node APIs for C API, support for > > > > > > > > > > > > > > > > PCKS12 Keystores, > > and > > > > > > better > > > > > > > > > > > > > > > > procedure > > > > > > > > > > > > > > > > for the upgrade of servers from 3.4 to > > > > > > > > > > > > > > > > 3.5. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The full release notes is available at: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801&version=12345243 > > > > > > > > > > > > > > > > > *** Please download, test and vote by > > > > > > > > > > > > > > > > October 2nd 2019, > > > > 23:59 > > > > > > > > > UTC+0. > > > > > > > > > > > > > *** > > > > > > > > > > > > > > > > Source files: > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://people.apache.org/~eolivelli/zookeeper-3.5.6-candidate-2 > > > > > > > > > > > > > > > > > > > > > > Maven staging repo: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapachezookeeper-1042/ > > > > > > > > > > > > > > > > > The release candidate tag in git to be > > > > > > > > > > > > > > > > voted upon: > > > > > > > > > > > > > > > > > > release-3.5.6-rc2 > > https://github.com/apache/zookeeper/tree/release-3.5.6-rc2 > > > > > > > > > > > > > > > > > ZooKeeper's KEYS file containing PGP > > > > > > > > > > > > > > > > keys we use to sign > > > > the > > > > > > > > > > > > release: > > > > > > > > > > > > > > > > https://www.apache.org/dist/zookeeper/KEYS > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Should we release this candidate? > > > > > > > > > > > > > > > > Enrico Olivelli > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >