From dev-return-60850-archive-asf-public=cust-asf.ponee.io@phoenix.apache.org Wed Apr 22 16:07:21 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id E3174180181 for ; Wed, 22 Apr 2020 18:07:20 +0200 (CEST) Received: (qmail 56728 invoked by uid 500); 22 Apr 2020 16:07:20 -0000 Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list dev@phoenix.apache.org Received: (qmail 56716 invoked by uid 99); 22 Apr 2020 16:07:19 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 22 Apr 2020 16:07:19 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 41A471812A5 for ; Wed, 22 Apr 2020 16:07:10 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.2 X-Spam-Level: X-Spam-Status: No, score=-0.2 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id 0wyVlF4UT1_M for ; Wed, 22 Apr 2020 16:07:08 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2607:f8b0:4864:20::431; helo=mail-pf1-x431.google.com; envelope-from=andrew.purtell@gmail.com; receiver= Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id AF7B97FB3A for ; Wed, 22 Apr 2020 16:07:07 +0000 (UTC) Received: by mail-pf1-x431.google.com with SMTP id 18so95528pfv.8 for ; Wed, 22 Apr 2020 09:07:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=content-transfer-encoding:from:mime-version:subject:date:message-id :references:in-reply-to:to; bh=Bu1B3LzvAPMCCjp9u0NPczWdZUBjENse31GZJ+YNyEM=; b=kiK2lTfcDpbITCS5ycorl6o3L3z0NiTR4gb1O76YKmA6SB49HFTdcqBqaPHGKhJ4yq v8KHwtv2j+kh0kKgR3TQ6r7drOKzgUtlH9UnsidcjmiQETNbwWpxeTjT4b3vjppbvJd/ QSD4akpnie0GUtKg0clCdSspecUINI3nWqj7/gEFg+ekk2HKCXD+o2pJZoiWwp+TfxZ0 5IdsdjxFfkYfEVpdEVwn0qgfOnzg11jdXXxT2akicM6IuufDNH/cmJLsZt5YDAOuyEpd LPNPkUeNwD8Xq3wHt+KsRR3R3mveZdY0DnYepijPWH2cHiLiSWBi7M3cD3iAhZE+pBYf 6r2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version :subject:date:message-id:references:in-reply-to:to; bh=Bu1B3LzvAPMCCjp9u0NPczWdZUBjENse31GZJ+YNyEM=; b=dPFN3cuASZ61qV21rLbpAwg9dHsmrZizJR88q/YxIL/1GI3iDB8xpJWmfAzgzPpkI2 f64mL3rAZY0vztZEgt5if+5exv7uKWJRpRn9Up48zo83JZ+mQswi2tfQhCMZLYritzAf 1TIdjzj3bcSlguJ6jbTUr+c+TAkI/wAfitDUZTvDU4+I5sQ5+nXVmo+7hisWWfGIm/iU lzzlCRt80w42eEMinIuTiHV4fpcWeKt/T2RM8J9/1sntIQMLMnBjJSlubwdHv6f+qSim vnLkZQ9lY1epOrm7vp2nWMkTRbUJ2FVYz7L8OzixKJf5z45kW2ZqiX5iFvRMKTO4GDcb clfQ== X-Gm-Message-State: AGi0Pua6ZQhPbSKu8b3+R0IVK+2yayEbALe2PfxAxd2MHSwHJIFUxTSW tnjPVm5MMwaJONdx/dq/ZxkaPET3 X-Google-Smtp-Source: APiQypJAHkOjKeyR+hr4uxarnMLAkOYVnBRDJmHeDVrkm2mBrn1UYmTDHc3xDudlNPZGHxaRkNYNtg== X-Received: by 2002:a63:d24f:: with SMTP id t15mr10269541pgi.214.1587571619186; Wed, 22 Apr 2020 09:06:59 -0700 (PDT) Received: from [192.168.1.19] (cpe-76-174-21-114.socal.res.rr.com. [76.174.21.114]) by smtp.gmail.com with ESMTPSA id o7sm6033925pfg.74.2020.04.22.09.06.58 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 22 Apr 2020 09:06:58 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable From: Andrew Purtell Mime-Version: 1.0 (1.0) Subject: Re: Are we embracing Java 8? Date: Wed, 22 Apr 2020 09:06:57 -0700 Message-Id: References: In-Reply-To: To: dev@phoenix.apache.org X-Mailer: iPhone Mail (17E262) Yes, a real cluster, ideally a real application. With some stress. Some scal= e too if it can be managed.=20 > On Apr 22, 2020, at 6:10 AM, Istvan Toth wrote: >=20 > =EF=BB=BFAgreed, making this change only makes sense if we bump the source= version, > and also compile Java 8 bytecode, which precludes Java7 support. > Cross-compiling to Java 7 would defeat the whole point of bumping the Java= > version. >=20 > Re the test issue: >=20 > I am pretty sure that If we bump the Java version in the 4.x POM, then run= > the standard test suite, it will test the very java compiler/runtime > combination that you are referring to. > Java 8 will be used to compile the Phoenix classes, The HBase and Hadoop > artifacts that maven pulls in will be Compiled with 1.7, and the Java > Runtime that runs all of this will be Java 8. >=20 > What testing do you have in mind that would be more thorough than that ? > We can run the test suite on a real HBase 1.3 cluster as well (most of it)= , > but since we have but one test suite, the only difference would be that > HBase and Java run in different JVMs. > (Of course, it would be useful, and probably worth doing, but it would be > fundamentally the same Java/JVM combination) >=20 > regards > Istvan >=20 >> On Wed, Apr 22, 2020 at 7:29 AM Andrew Purtell = >> wrote: >>=20 >> Yes, what I was trying to say is even if you don't adopt 8 language >> features and specify a source level of 7 when compiling the binaries, the= re >> will be other runtime issues. And so sure there's no reason to hold back o= n >> source level changes. The runtime compatibility story will change >> overnight. >>=20 >> Regarding testing... Unless you think that someone won't download a HBase= >> binary release (1.x will be compiled with 7), and then a future Phoenix >> binary release (compiled with 8), and then try to combine them, you shoul= d >> probably test that combination. (Smile.) >>=20 >>=20 >>>> On Apr 21, 2020, at 9:44 PM, Istvan Toth wrote: >>>=20 >>> =EF=BB=BFHi! >>>=20 >>> Actually, the convenience binaries would be the least of a (hypothetical= ) >>> Java7 user's problem. >>> The whole point of moving 4.x to Java8 would be to enable the usage of >>> Java8 features in the project, >>> so almost immediately the 4.x branch wouldn't even compile on 4.x . >>>=20 >>> I would imagine that any installation still stuck on Java7 would be a >>> thoroughly legacy system where updating Phoenix is not a priority. >>>=20 >>> I agree with the importance of having to thoroughly and prominently >>> document such a change. >>>=20 >>> I am not sure that additional testing would be needed to test the >> HBase1.x >>> compiled with Java7 case, >>> AFAICT our IT suite would test exactly that case, as the HBase1.x >> artifacts >>> in Maven that the tests pull in are compiled on Java7. >>>=20 >>> Miangliang is certainly not the only one who wants to use Java8 features= . >>> This is a limitation that the old hands may be used to, but >>> is a pain point for bright-eyed new contributors. I know I've had to ask= >> a >>> new dev to rewrite Java8 specific code multiple times, >>> not to mention having to rewrite my code on backport when I didn't >> realize >>> that the feature I used was Java8 specific. >>>=20 >>> I can see these three outcomes: >>>=20 >>> a. Switch 4.x to Java 8 now, and deal with the Java8->Java7 porting >> issues >>> on backporting to 4.15.x >>> b. Switch 4.x to Java 8 on the release on 4.16, thereby saving the >>> backporting issues on 4.15.x >>> c. Postpone switching to Java8 until HBase 1.x goes out of support. >>>=20 >>> I know that no-one wants to commit to release dates, but do we even plan= >> to >>> have a 4.16 release soon-ish ? >>> Based on historical release cadence, it could be anytime from next month= >> to >>> next year. >>>=20 >>> Having more or less established that no-one who is active on the dev lis= t >>> cares about Java 7 for his/her own use, >>> how should we go forward with this ? >>>=20 >>> Is this something that we should have a formal vote on ? >>>=20 >>> Disclaimer: my employer doesn't support recent Phoenix 4.x versions, so >> my >>> opinions may be tinted by that. >>>=20 >>> regards >>> Istvan >>>=20 >>>> On Wed, Apr 22, 2020 at 3:50 AM Andrew Purtell >> wrote: >>>>=20 >>>> Just to be super clear: >>>>=20 >>>>> Compile the code with Java 8, then if someone tries to install the >>>> resulting binaries into a HBase convenience binary release 1.x, which >> will >>>> have been compiled with Java 7, the regionserver will abort. >>>>=20 >>>> ... if the code is running on Java 7. >>>>=20 >>>> I don't know if anyone is actually still running Java 7 anywhere, and >>>> wanting to use HBase and Phoenix on that runtime, but it's possible. We= >>>> haven't tried to move up minimum Java version on HBase 1.x because it's= >>>> impossible to know who is using it where, and originally it was release= d >>>> for/on 7. You can decide it is unlikely enough to move forward, but >> should >>>> at least document the implications somewhere on the release page. >>>>=20 >>>> I'm pretty sure HBase compiled with 7 and Phoenix compiled with 8, >> running >>>> on 8 or later, will be stable, but this configuration obviously should >> be >>>> rigorously tested if you plan to move up. >>>>=20 >>>> >>>>=20 >>>>=20 >>>> On Tue, Apr 21, 2020 at 6:44 PM Andrew Purtell >>>> wrote: >>>>=20 >>>>> The main problem you will face is the convenience binaries. >>>>>=20 >>>>> If you are planning to move to Java 8, for 4.x branches, then you will= >> no >>>>> longer be able to make binary convenience releases compatible with any= >>>>> HBase 1.x convenience binary, even if you don't adopt any Java 8 >> language >>>>> features. Compile the code with Java 8, then if someone tries to >> install >>>>> the resulting binaries into a HBase convenience binary release 1.x, >> which >>>>> will have been compiled with Java 7, the regionserver will abort. >>>>>=20 >>>>> Java 8 is backwards compatible with Java 7. That is, if you compile >> with >>>>> Java 7 but run on a Java 8 runtime, all is good, even linkage in the >> JRE. >>>>>=20 >>>>> Java 7 is not forwards compatible with Java 8, especially with respect= >> to >>>>> JRE internals. What that means is if you compile something with Java 8= , >>>>> even if you are not using Java 8 language features, you won't be able >> to >>>>> run it on a Java 7 runtime. Notably, there are linkage errors in the >>>>> j.u.concurrent package, which as you know both Phoenix and HBase use >>>>> extensively. I suppose this doesn't matter if you are adopting Java 8 >>>>> language features anyway. >>>>>=20 >>>>> Seems a big deal to move to source only releases for 4.x, but that is >> an >>>>> option. >>>>>=20 >>>>>=20 >>>>>> On Fri, Apr 17, 2020 at 10:18 PM Istvan Toth >> wrote: >>>>>=20 >>>>>> Hi! >>>>>>=20 >>>>>> Given that in a few months we will be in the awkward position where >>>> HBase >>>>>> 1.x mandates Java 7 support, but even OpenJDK will have Java 7 EOLed,= >>>> this >>>>>> may actually be a good time to revisit the decision to keep 4.x on >> Java >>>> 7. >>>>>>=20 >>>>>> All supported HBase versions support Java8. (Just checked) >>>>>>=20 >>>>>> Do we know of any major 4.x users (looking at SFDC mostly), who are >>>> still >>>>>> running HBase/Phoenix with Java 7, and plan to stay - and update >>>> Phoenix >>>>>> beyond 4.15.x - on it ? >>>>>>=20 >>>>>> How about bumping the Java requirement on 4.x to Java8 on with the >>>> release >>>>>> of 4.16 ? >>>>>>=20 >>>>>> This way we wouldn't take on the backport problems with 4.15.x, but >> we >>>>>> would be able to use the new functionality in a reasonably timely >>>> manner. >>>>>>=20 >>>>>> regards >>>>>> Istvan >>>>>>=20 >>>>>> On Sat, Apr 18, 2020 at 1:17 AM Mingliang Liu >>>> wrote: >>>>>>=20 >>>>>>> Thanks for the comments, Geoffrey. >>>>>>>=20 >>>>>>> I guess the option 3 is not preferred by anyone. For option 2, I did= >>>> not >>>>>>> know community has discussed on this and the conclusion still holds >>>>>> today. >>>>>>> Glad to know. If timing is good, someone can reopen this conversatio= n >>>>>>> later. >>>>>>>=20 >>>>>>> On Fri, Apr 17, 2020 at 1:32 PM Geoffrey Jacoby = >>>>>>> wrote: >>>>>>>=20 >>>>>>>> Mingliang, >>>>>>>>=20 >>>>>>>> The topic's come up before, and in the past the conclusion has been= >>>> to >>>>>>> keep >>>>>>>> our Java requirements in line with the HBase dependency of a given >>>>>>> branch. >>>>>>>> Since HBase 1.x supports Java 7, and HBase compatibility guidelines= >>>>>> don't >>>>>>>> allow for making JDK requirements more strict within a major >>>> release, >>>>>>> that >>>>>>>> meant keeping Java 7 support on the 4.x branches which are of cours= e >>>>>>> based >>>>>>>> on HBase 1.x. (And I don't see the 4.x line going away anytime >>>> soon.) >>>>>>>>=20 >>>>>>>> We can always reopen that conversation about JDK support in light o= f >>>>>> the >>>>>>>> upcoming EOL, but so long as the requirement for Java 7 support is >>>>>>> present, >>>>>>>> I agree with Istvan that I wouldn't want large-scale changes betwee= n >>>>>>> master >>>>>>>> and 4.x based on JDK differences, because it's more work on both >>>> patch >>>>>>>> authors and committers the more they diverge. So I don't favor >>>> Option >>>>>> 2. >>>>>>>>=20 >>>>>>>> Geoffrey >>>>>>>>=20 >>>>>>>> On Fri, Apr 17, 2020 at 12:27 PM Mingliang Liu >>>>>>> wrote: >>>>>>>>=20 >>>>>>>>> Hi, >>>>>>>>>=20 >>>>>>>>> I filed PHOENIX-5855 < >>>>>>> https://issues.apache.org/jira/browse/PHOENIX-5855 >>>>>>>>>=20 >>>>>>>>> to >>>>>>>>> make the code more Java 8. This may apply to master branch only >>>> and >>>>>>>> Istvan >>>>>>>>> Toth expressed concern about backporting conflicts. >>>>>>>>>=20 >>>>>>>>> I guess this is the trade-off between embracing newer Java >>>> platform >>>>>>>> (Java 7 >>>>>>>>> is EOL and will not be supported next year) and the effort of >>>>>> resolving >>>>>>>>> conflict if any when backporting. >>>>>>>>>=20 >>>>>>>>> The options are: >>>>>>>>>=20 >>>>>>>>> 1. get stuck in Java 7 for both master and all old branches. We >>>>>> are >>>>>>>>> basically on this approach as I see Java 8 is used very >>>>>> sparingly in >>>>>>>>> master >>>>>>>>> branch >>>>>>>>> 2. use Java 8 in master branch extensively and take care of >>>> minor >>>>>>>>> conflicts if any for all 4.x branches. Patch author can provide >>>>>>>> separate >>>>>>>>> patch if conflict is major, or make changes with less conflict. >>>>>>>>> 3. bump 4.16 (or 4.15.1?) to Java 8. Backporting to older >>>>>> branches >>>>>>>> will >>>>>>>>> still need some manual fix as above. >>>>>>>>>=20 >>>>>>>>> I think it is the right time for option 2 or 3. Thoughts? >>>>>>>>>=20 >>>>>>>>> Thanks, >>>>>>>>> -- >>>>>>>>> L >>>>>>>>>=20 >>>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>> -- >>>>>>> L >>>>>>>=20 >>>>>>=20 >>>>>=20 >>>>>=20 >>>>> -- >>>>> Best regards, >>>>> Andrew >>>>>=20 >>>>> Words like orphans lost among the crosstalk, meaning torn from truth's= >>>>> decrepit hands >>>>> - A23, Crosstalk >>>>>=20 >>>>=20 >>>>=20 >>>> -- >>>> Best regards, >>>> Andrew >>>>=20 >>>> Words like orphans lost among the crosstalk, meaning torn from truth's >>>> decrepit hands >>>> - A23, Crosstalk >>>>=20 >>=20