Return-Path: X-Original-To: apmail-hadoop-common-dev-archive@www.apache.org Delivered-To: apmail-hadoop-common-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CE3CA11E75 for ; Fri, 20 Jun 2014 18:02:49 +0000 (UTC) Received: (qmail 66164 invoked by uid 500); 20 Jun 2014 18:02:47 -0000 Delivered-To: apmail-hadoop-common-dev-archive@hadoop.apache.org Received: (qmail 66091 invoked by uid 500); 20 Jun 2014 18:02:47 -0000 Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-dev@hadoop.apache.org Received: (qmail 66078 invoked by uid 99); 20 Jun 2014 18:02:47 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 20 Jun 2014 18:02:47 +0000 X-ASF-Spam-Status: No, hits=0.6 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS,URI_HEX X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of rarecactus@gmail.com designates 209.85.212.177 as permitted sender) Received: from [209.85.212.177] (HELO mail-wi0-f177.google.com) (209.85.212.177) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 20 Jun 2014 18:02:44 +0000 Received: by mail-wi0-f177.google.com with SMTP id r20so1235773wiv.10 for ; Fri, 20 Jun 2014 11:02:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type:content-transfer-encoding; bh=Z3tZA6ZsFPlvevZtjFmQP5wtitfzMYl+FohvmsFx1/k=; b=j6QxKvWRqscqZg3WFsmgjhzRMIrYFIpXMNqiIUrYuMR3I5hLfDwuxmdrvIEHZgLPSj ycYMVzb+fd/N1O+0nsYyaXtzemzbbl05CM2Z6qBs/HVDDe82nTP1WGE9TLFwra7ULLBL WUcwEhtxbGpM0+lICdRb/M8r/sflPkBqGJMQKIUoK8mNk8KZ9zEMOaHZOBjvzG6AoahP bueZk6Cei9hPtImBnpfRS87r8nkqsQpyRM4qHPO5oqcVPHlhnZV0ZF3V9alUlf7Nh/p+ uBy2m/nACNY2lz8OPcP1U0mEiHttjtwo5bXxbVmpS12L4he4qH4shll3s2sOTiQEJcmc D84Q== MIME-Version: 1.0 X-Received: by 10.194.110.10 with SMTP id hw10mr6371978wjb.81.1403287340400; Fri, 20 Jun 2014 11:02:20 -0700 (PDT) Sender: rarecactus@gmail.com Received: by 10.194.242.35 with HTTP; Fri, 20 Jun 2014 11:02:20 -0700 (PDT) In-Reply-To: References: <92FD1DEAAC387041B70D5237A1BE3A1D0CFC835813@MX33A.corp.emc.com> <92FD1DEAAC387041B70D5237A1BE3A1D0CFF46A2AD@MX33A.corp.emc.com> Date: Fri, 20 Jun 2014 11:02:20 -0700 X-Google-Sender-Auth: pNAl2wCTNLm6j2srm7dVAWbhTqw Message-ID: Subject: Re: Plans of moving towards JDK7 in trunk From: Colin McCabe To: Hadoop Common Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org I think the important thing to do right now is to ensure our code works with jdk8. This is similar to the work we did last year to fix issues that cropped up with jdk7. There were just a few minor things like the error in which it ran unit tests that caused issues. I don't think it's out of the question to bump minVersion directly from 1.6 to 1.8. Java 7 is EOL in less than a year, after all... at least according to the current roadmap here: http://www.oracle.com/technetwork/java/eol-135779.html We should try to do things in a way that causes the least disruption for users upgrading clusters, if possible... cheers, Colin On Thu, Jun 19, 2014 at 3:31 PM, Andrew Purtell wrote= : > There are a number of security (algorithm, not vulnerability) and > performance improvements that landed in 8, not 7. As a runtime for the > performance conscious, it might be recommendable. I've come across GC > issues in 6 or 7 where, talking with some Java platform folks, the first > suggested course of action is try again with 8. Would be be more of the > current moment if this discussion was about setting guidelines that > prescribe when and when not to use 8+ language features, or concurrency > library improvements that rely on intrinsics only available in the 8 > runtime? Has the Java 6 ship sailed? Just set the minimum supported JDK a= nd > runtime at 7 at next release? Use of the <> operator or multicatch wouldn= 't > and shouldn't need be debated, they are quite minor. On the other hand, I > would imagine discussion and debate on what 8+ language features might be > useful to use at some future time could be a lively one. > > > > On Wed, Jun 18, 2014 at 3:03 PM, Colin McCabe > wrote: > >> In CDH5, Cloudera encourages people to use JDK7. JDK6 has been EOL >> for a while now and is not something we recommend. >> >> As we discussed before, everyone is in favor of upgrading to JDK7. >> Every cluster operator of a reasonably modern Hadoop should do it.... >> whatever distro or release you run. As developers, we run JDK7 as >> well. >> >> I'd just like to see a plan for when branch-2 (or some other branch) >> will create a stable release that drops support for JDK1.6. If we >> don't have such a plan, I feel like it's too early to talk about this >> stuff. >> >> If we drop support for 1.6 in trunk but not in branch-2, we are >> fragmenting the project. People will start writing unreleaseable code >> (because it doesn't work on branch-2) and we'll be back to the bad old >> days of Hadoop version fragmentation that branch-2 was intended to >> solve. Backports will become harder. The biggest problem is that >> trunk will start to depend on libraries or Maven plugins that branch-2 >> can't even use, because they're JDK7+-only. >> >> Steve wrote: "if someone actually did file a bug on something on >> branch-2 which didn't work on Java 6 but went away on Java7+, we'd >> probably close it as a WORKSFORME". >> >> Steve, if this is true, we should just bump the minimum supported >> version for branch-2 to 1.7 today and resolve this. If we truly >> believe that there are no issues here, then let's just decide to drop >> 1.6 in a specific future release of Hadoop 2. If there are issues >> with releasing JDK1.7+ only code, then let's figure out what they are >> before proceeding. >> >> best, >> Colin >> >> >> On Wed, Jun 18, 2014 at 1:41 PM, Sandy Ryza >> wrote: >> > We do release warnings when we are aware of vulnerabilities in our >> > dependencies. >> > >> > However, unless I'm grossly misunderstanding, the vulnerability that y= ou >> > point out is not a vulnerability within the context of our software. >> > Hadoop doesn't try to sandbox within JVMs. In a secure setup, any JV= M >> > running non-trusted user code is running as that user, so "breaking ou= t" >> > doesn't offer the ability to do anything malicious. >> > >> > -Sandy >> > >> > On Wed, Jun 18, 2014 at 1:30 PM, Ottenheimer, Davi < >> Davi.Ottenheimer@emc.com >> >> wrote: >> > >> >> Andrew, >> >> >> >> >> >> >> >> =E2=80=9CI don't see any point to switching=E2=80=9D is an interestin= g perspective, >> given >> >> the well-known risks of running unsafe software. Clearly customer bes= t >> >> interest is stability. JDK6 is in a known unsafe state. The longer >> anyone >> >> delays the necessary transition to safety the longer the door is left >> open >> >> to predictable disaster. >> >> >> >> >> >> >> >> You also said "we still test and support JDK6". I searched but have n= ot >> >> been able to find Cloudera critical security fixes for JDK6. >> >> >> >> >> >> >> >> Can you clarify, for example, Java 6 Update 51 for CVE-2013-2465? In >> other >> >> words, did you release to your customers any kind of public alert or >> >> warning of this CVSS 10.0 event as part of your JDK6 support? >> >> >> >> >> >> >> >> http://www.cvedetails.com/cve/CVE-2013-2465/ >> >> >> >> >> >> >> >> If you are not releasing your own security fixes for JDK6 post-EOL wo= uld >> >> it perhaps be safer to say Cloudera is hands-off; neither supports, n= or >> >> opposes the known insecure and deprecated/unpatched JDK? >> >> >> >> >> >> >> >> I mentioned before in this thread the Oracle support timeline: >> >> >> >> >> >> >> >> - official public EOL (end of life) was more than a year ago >> >> >> >> - premier support ended more than six months ago >> >> >> >> - extended support may get critical security fixes until the end of 2= 016 >> >> >> >> >> >> >> >> Given this timeline, does Cloudera officially take responsibility for >> >> Hadoop customer safety? Are you going to be releasing critical securi= ty >> >> fixes to a known unsafe JDK? >> >> >> >> >> >> >> >> Davi >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > -----Original Message----- >> >> >> >> > From: Andrew Wang [mailto:andrew.wang@cloudera.com] >> >> >> >> > Sent: Wednesday, June 18, 2014 12:33 PM >> >> >> >> > To: common-dev@hadoop.apache.org >> >> >> >> > Subject: Re: Plans of moving towards JDK7 in trunk >> >> >> >> > >> >> >> >> > Actually, a lot of our customers are still on JDK6, so if anything, >> its >> >> popularity >> >> >> >> > hasn't significantly decreased. We still test and support JDK6 for >> CDH4 >> >> and >> >> >> >> > CDH5. The claim that branch-2 is effectively JDK7 because no one >> supports >> >> >> >> > JDK6 is untrue. >> >> >> >> > >> >> >> >> > One issue with your proposal is that java 7+ libraries can have >> >> incompatible >> >> >> >> > APIs compared to their java 6 versions. Guava moves very quickly wi= th >> >> regard >> >> >> >> > to the deprecate+remove cycle. This means branch-2 and trunk >> divergence, >> >> >> >> > as we're stuck using different Guava APIs to do the same thing. >> >> >> >> > >> >> >> >> > No one's arguing against moving to Java 7+ in trunk eventually, but >> >> there isn't >> >> >> >> > a clear plan for a trunk-based release. I don't see any point to >> >> switching trunk >> >> >> >> > over until that's true, for the aforementioned reasons. >> >> >> >> > >> >> >> >> > Best, >> >> >> >> > Andrew >> >> >> >> > >> >> >> >> > >> >> >> >> > On Wed, Jun 18, 2014 at 12:08 PM, Steve Loughran >> >> >> >> > > >> >> >> >> > wrote: >> >> >> >> > >> >> >> >> > > I also think we need to recognise that its been three months sinc= e >> >> >> >> > > that last discussion, and Java 6 has not suddenly burst back into >> >> >> >> > > popularity >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > - nobody providing commercial support for Hadoop is offering >> >> branch-2 >> >> >> >> > > support on Java 6 AFAIK >> >> >> >> > > - therefore, nobody is testing it at scale except privately, a= nd >> >> they >> >> >> >> > > aren't reporting bugs if they are >> >> >> >> > > - if someone actually did file a bug on something on branch-2 >> which >> >> >> >> > > didn't work on Java 6 but went away on Java7+, we'd probably >> close >> >> >> >> > > it as a >> >> >> >> > > WORKSFORME >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > whether we acknowledge it or not, Hadoop 2.x is now really Java 7= +. >> >> >> >> > > >> >> >> >> > > We do all agree that hadoop 3 will not be java 6, so the only iss= ue >> is >> >> >> >> > > "when and how to make that transition". >> >> >> >> > > >> >> >> >> > > That patch of mine just makes it possible to do today. >> >> >> >> > > >> >> >> >> > > I have actually jumped to Java7 in the slider project, and actual= ly >> >> >> >> > > being using Java 8 and twill; the new language features there are >> >> >> >> > > significant and would be great to use in Hadoop *at some point in >> the >> >> >> >> > > future* >> >> >> >> > > >> >> >> >> > > For Java 7 though, based on that experience, the language changes >> are >> >> >> >> > > convenient but not essential >> >> >> >> > > >> >> >> >> > > - try-with-resources simply swallows close failures without th= e >> log >> >> >> >> > > integration we have with IOUtils.closeStream(), so shoudn't be >> used >> >> in >> >> >> >> > > hadoop core anyway. >> >> >> >> > > - string based switching: convenient, but not critical >> >> >> >> > > - type inference on template constructors. Modern IDEs handle = the >> >> pain >> >> >> >> > > anyway >> >> >> >> > > >> >> >> >> > > The only feature I like is multi-catch and typed rethrow >> >> >> >> > > >> >> >> >> > > catch(IOException | ExitException e) { log.warn(e.toString(); >> throw >> >> >> >> > > e; } >> >> >> >> > > >> >> >> >> > > this would make "e" look like Exception, but when rethrown go bac= k >> to >> >> >> >> > > its original type. >> >> >> >> > > >> >> >> >> > > This reduces duplicate work, and is the bit l actually value. Is = it >> >> >> >> > > enough to justify making code incompatible across branches? No. >> >> >> >> > > >> >> >> >> > > So i'm going to propose this, and would like to start a vote on i= t >> >> >> >> > > soon >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > 1. we parameterize java versions in the POMs on all branches, >> with >> >> >> >> > > separate JDK versions and Java language >> >> >> >> > > 2. branch-2: java-6-language and JDK-6 minimum JDK >> >> >> >> > > 3. trunk: java-6-language and JDK-7 minimum JDK >> >> >> >> > > >> >> >> >> > > This would guarantee that none of the java 7 language features we= nt >> >> >> >> > > in, but we could move trunk up to java 7+ only libraries (jersey, >> >> >> >> > > guava). Adopting >> >> >> >> > > JDK7 features then becomes no more different from adopting java7+ >> >> >> >> > > libraries: those bits of code that have moved can't be backported= . >> >> >> >> > > >> >> >> >> > > -Steve >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > On 17 June 2014 22:08, Andrew Wang > > >> andrew.wang@cloudera.com>> >> >> >> >> > wrote: >> >> >> >> > > >> >> >> >> > > > Reviving this thread, I noticed there's been a patch and +1 on >> >> >> >> > > > HADOOP-10530, and I don't think we actually reached a conclusio= n. >> >> >> >> > > > >> >> >> >> > > > I (and others) have expressed concerns about moving to JDK7 for >> >> trunk. >> >> >> >> > > > Summarizing a few points: >> >> >> >> > > > >> >> >> >> > > > - We can't move to JDK7 in branch-2 because of compatibility >> >> >> >> > > > - branch-2 is currently the only Hadoop release vehicle, there = are >> >> >> >> > > > no >> >> >> >> > > plans >> >> >> >> > > > for a trunk-based Hadoop 3 >> >> >> >> > > > - Introducing JDK7-only APIs in trunk will increase divergence >> with >> >> >> >> > > > branch-2 and make backports harder >> >> >> >> > > > - Almost all developers care only about branch-2, since it is t= he >> >> >> >> > > > only release vehicle >> >> >> >> > > > >> >> >> >> > > > With this in mind, I struggle to see any upsides to introducing >> >> >> >> > > > JDK7-only APIs to trunk. Please let's not do anything on >> >> >> >> > > > HADOOP-10530 or related until we agree on this. >> >> >> >> > > > >> >> >> >> > > > Thanks, >> >> >> >> > > > Andrew >> >> >> >> > > > >> >> >> >> > > > >> >> >> >> > > > On Mon, Apr 14, 2014 at 3:31 PM, Steve Loughran >> >> >> >> > > > > >> >> >> >> > > > wrote: >> >> >> >> > > > >> >> >> >> > > > > On 14 April 2014 17:46, Andrew Purtell > >> > wrote: >> >> >> >> > > > > >> >> >> >> > > > > > How well is trunk tested? Does anyone deploy it with real >> >> >> >> > > applications >> >> >> >> > > > > > running on top? When will the trunk codebase next be the ba= sis >> >> >> >> > > > > > for a production release? An impromptu diff of hadoop-commo= n >> >> >> >> > > > > > trunk against >> >> >> >> > > > > > branch-2 as of today is 38,625 lines. Can they be said to b= e >> the >> >> >> >> > > > > > same animal? I ask because any disincentive toward putting >> code >> >> >> >> > > > > > in trunk >> >> >> >> > > is >> >> >> >> > > > > > beside the point, if the only target worth pursuing today i= s >> >> >> >> > > > > > branch-2 unless one doesn't care if the code is released fo= r >> >> >> >> > production use. >> >> >> >> > > > > > Questions on whither JDK6 or JDK7+ (or JRE6 versus JRE7+) o= nly >> >> >> >> > > > > > matter >> >> >> >> > > > for >> >> >> >> > > > > > the vast majority of Hadoopers if talking about branch-2. >> >> >> >> > > > > > >> >> >> >> > > > > > >> >> >> >> > > > > I think its partly a timescale issue; its also because the 1-= 2 >> >> >> >> > > transition >> >> >> >> > > > > was so significant, especially at the YARN layer, that it's >> still >> >> >> >> > > taking >> >> >> >> > > > > time to trickle through. >> >> >> >> > > > > >> >> >> >> > > > > If you do want code to ship this year, branch-2 is where you = are >> >> >> >> > > > > going >> >> >> >> > > to >> >> >> >> > > > > try and get it in -and like you say, that's where things get >> tried >> >> >> >> > > > > in >> >> >> >> > > the >> >> >> >> > > > > field. At the same time, the constraints of stability are >> holding >> >> >> >> > > > > us >> >> >> >> > > back >> >> >> >> > > > > -already-. >> >> >> >> > > > > >> >> >> >> > > > > I don't see why we should have such another major 1-2 transit= ion >> >> >> >> > > > > in >> >> >> >> > > > future; >> >> >> >> > > > > the rate that Arun is pushing out 2.x releases its almost bac= k >> to >> >> >> >> > > > > the >> >> >> >> > > > 0.1x >> >> >> >> > > > > timescale -though at that point most people were fending for >> >> >> >> > > > > themselves >> >> >> >> > > > and >> >> >> >> > > > > expectations of stability were less. We do want smaller versi= on >> >> >> >> > > > increments >> >> >> >> > > > > in future, which branch-2 is -mostly- delivering. >> >> >> >> > > > > >> >> >> >> > > > > While Java 7 doesn't have some must-have features, Java 8 is = a >> >> >> >> > > > significant >> >> >> >> > > > > improvement in the language, and we should be looking ahead t= o >> >> >> >> > > > > that, >> >> >> >> > > > maybe >> >> >> >> > > > > even doing some leading-edge work on the side, so the same >> >> >> >> > > > > discussion doesn't come up in two years time when java 7 goes >> EOL. >> >> >> >> > > > > >> >> >> >> > > > > >> >> >> >> > > > > -steve >> >> >> >> > > > > >> >> >> >> > > > > (personal opinions only, etc, ) >> >> >> >> > > > > >> >> >> >> > > > > >> >> >> >> > > > > > >> >> >> >> > > > > > On Mon, Apr 14, 2014 at 9:22 AM, Colin McCabe < >> >> >> >> > > cmccabe@alumni.cmu.edu >> >> >> >> > > > > > >wrote: >> >> >> >> > > > > > >> >> >> >> > > > > > > I think the bottom line here is that as long as our stabl= e >> >> >> >> > > > > > > release uses JDK6, there is going to be a very, very stro= ng >> >> >> >> > > > > > > disincentive to put any code which can't run on JDK6 into >> >> trunk. >> >> >> >> > > > > > > >> >> >> >> > > > > > > Like I said earlier, the traditional reason for putting >> >> >> >> > > > > > > something >> >> >> >> > > in >> >> >> >> > > > > > > trunk but not the stable release is that it needs more >> testing. >> >> >> >> > > If a >> >> >> >> > > > > > > stable release that drops support for JDK6 is more than a >> year >> >> >> >> > > away, >> >> >> >> > > > > > > does it make sense to put anything in trunk like that? W= hat >> >> >> >> > > > > > > might need more than a year of testing? Certainly not >> changes >> >> >> >> > > > > > > to LocalFileSystem to use the new APIs. I also don't thi= nk >> an >> >> >> >> > > > > > > upgrade >> >> >> >> > > > to >> >> >> >> > > > > > > various libraries qualifies. >> >> >> >> > > > > > > >> >> >> >> > > > > > > It might be best to shelve this for now, like we've done = in >> >> >> >> > > > > > > the >> >> >> >> > > past, >> >> >> >> > > > > > > until we're ready to talk about a stable release that >> requires >> >> >> >> > > JDK7+. >> >> >> >> > > > > > > At least that's my feeling. >> >> >> >> > > > > > > >> >> >> >> > > > > > > If we're really desperate for the new file APIs JDK7 >> provides, >> >> >> >> > > > > > > we could consider using loadable modules for it in branch= -2. >> >> >> >> > > > > > > This is similar to how we provide JNI versions of certain >> >> >> >> > > > > > > things on certain platforms, without dropping support for >> the >> >> other >> >> >> >> > platforms. >> >> >> >> > > > > > > >> >> >> >> > > > > > > best, >> >> >> >> > > > > > > Colin >> >> >> >> > > > > > > >> >> >> >> > > > > > > On Sun, Apr 13, 2014 at 10:39 AM, Raymie Stata < >> >> >> >> > > rstata@altiscale.com >> >> >> >> > > > > >> >> >> >> > > > > > > wrote: >> >> >> >> > > > > > > > There's an outstanding question addressed to me: "Are >> there >> >> >> >> > > > > particular >> >> >> >> > > > > > > > features or new dependencies that you would like to >> >> >> >> > > > > > > > contribute >> >> >> >> > > (or >> >> >> >> > > > > see >> >> >> >> > > > > > > > contributed) that require using the Java 1.7 APIs?" Th= e >> >> >> >> > > > > > > > question misses the point: We'd figure out how to write >> >> >> >> > > > > > > > something we >> >> >> >> > > wanted >> >> >> >> > > > to >> >> >> >> > > > > > > > contribute to Hadoop against the APIs of Java4 if that'= s >> >> >> >> > > > > > > > what it >> >> >> >> > > > took >> >> >> >> > > > > > > > to get them into a stable release. And at current cour= se >> >> >> >> > > > > > > > and >> >> >> >> > > > speed, >> >> >> >> > > > > > > > that's how ridiculous things could get. >> >> >> >> > > > > > > > >> >> >> >> > > > > > > > To summarize, it seems like there's a vague consensus t= hat >> >> >> >> > > > > > > > it >> >> >> >> > > might >> >> >> >> > > > > be >> >> >> >> > > > > > > > okay to eventually allow the use of Java7 in trunk, but >> >> >> >> > > > > > > > there's >> >> >> >> > > no >> >> >> >> > > > > > > > decision. And there's been no answer to the concern th= at >> >> >> >> > > > > > > > even if >> >> >> >> > > > > such >> >> >> >> > > > > > > > dependencies were allowed in Java7, the only people usi= ng >> >> >> >> > > > > > > > them >> >> >> >> > > > would >> >> >> >> > > > > > > > be people who uninterested in getting their patches int= o a >> >> >> >> > > > > > > > stable release of Hadoop on any knowable timeframe, whi= ch >> >> >> >> > > > > > > > doesn't bode >> >> >> >> > > > well >> >> >> >> > > > > > > > for the ability to stabilize that Java7 code when it co= mes >> >> >> >> > > > > > > > time >> >> >> >> > > to >> >> >> >> > > > > > > > attempt to. >> >> >> >> > > > > > > > >> >> >> >> > > > > > > > I don't have more to add, so I'll go back to lurking. >> It'll >> >> >> >> > > > > > > > be interesting to see where we'll be standing a year fr= om >> >> now. >> >> >> >> > > > > > > > >> >> >> >> > > > > > > > On Sun, Apr 13, 2014 at 2:09 AM, Tsuyoshi OZAWA >> >> >> >> > > > > > > > > >> >> >> wrote: >> >> >> >> > > > > > > >> Hi, >> >> >> >> > > > > > > >> >> >> >> >> > > > > > > >> +1 for Karthik's idea(non-binding). >> >> >> >> > > > > > > >> >> >> >> >> > > > > > > >> IMO, we should keep the compatibility between JDK 6 an= d >> JDK >> >> >> >> > > > > > > >> 7 on >> >> >> >> > > > > both >> >> >> >> > > > > > > branch-1 >> >> >> >> > > > > > > >> and branch-2, because users can be using them. For fut= ure >> >> >> >> > > releases >> >> >> >> > > > > > that >> >> >> >> > > > > > > we can >> >> >> >> > > > > > > >> declare breaking compatibility(e.g. 3.0.0 release), we >> can >> >> >> >> > > > > > > >> use >> >> >> >> > > > JDK 7 >> >> >> >> > > > > > > >> features if we >> >> >> >> > > > > > > >> can get benefits. However, it can increase maintenance >> >> >> >> > > > > > > >> costs and >> >> >> >> > > > > > > distributes the >> >> >> >> > > > > > > >> efforts of contributions to maintain branches. Then, I >> >> >> >> > > > > > > >> think it >> >> >> >> > > is >> >> >> >> > > > > > > >> reasonable approach >> >> >> >> > > > > > > >> that we use limited and minimum JDK-7 APIs when we hav= e >> >> >> >> > > > > > > >> reasons >> >> >> >> > > we >> >> >> >> > > > > > need >> >> >> >> > > > > > > to use >> >> >> >> > > > > > > >> the features. >> >> >> >> > > > > > > >> By the way, if we start to use JDK 7 APIs, we should >> >> >> >> > > > > > > >> declare the >> >> >> >> > > > > basis >> >> >> >> > > > > > > >> when to use >> >> >> >> > > > > > > >> JDK 7 APIs on Wiki not to confuse contributors. >> >> >> >> > > > > > > >> >> >> >> >> > > > > > > >> Thanks, >> >> >> >> > > > > > > >> - Tsuyoshi >> >> >> >> > > > > > > >> >> >> >> >> > > > > > > >> On Wed, Apr 9, 2014 at 11:44 AM, Raymie Stata < >> >> >> >> > > > rstata@altiscale.com >> >> >> >> > > > > > >> >> >> >> > > > > > > wrote: >> >> >> >> > > > > > > >>>> It might make sense to try to enumerate the benefits= of >> >> >> >> > > > switching >> >> >> >> > > > > to >> >> >> >> > > > > > > >>>> Java7 APIs and dependencies. >> >> >> >> > > > > > > >>> >> >> >> >> > > > > > > >>> - Java7 introduced a huge number of language, >> byte-code, >> >> >> >> > > > > > > >>> API, >> >> >> >> > > > and >> >> >> >> > > > > > > >>> tooling enhancements! Just to name a few: >> >> >> >> > > > > > > >>> try-with-resources, >> >> >> >> > > > > newer >> >> >> >> > > > > > > >>> and stronger encyrption methods, more scalable >> concurrency >> >> >> >> > > > > > primitives. >> >> >> >> > > > > > > >>> See >> >> >> >> > > > > > > >>> >> http://www.slideshare.net/boulderjug/55-things-in-java-7 >> >> >> >> > > > > > > >>> >> >> >> >> > > > > > > >>> - We can't update current dependencies, and we can'= t >> add >> >> >> >> > > > > > > >>> cool >> >> >> >> > > > new >> >> >> >> > > > > > > ones. >> >> >> >> > > > > > > >>> >> >> >> >> > > > > > > >>> - Putting language/APIs aside, don't forget that a >> huge >> >> >> >> > > amount >> >> >> >> > > > of >> >> >> >> > > > > > > effort >> >> >> >> > > > > > > >>> goes into qualifying for Java6 (at least, I hope the >> folks >> >> >> >> > > > claiming >> >> >> >> > > > > > to >> >> >> >> > > > > > > >>> support Java6 are putting in such an effort :-). >> Wouldn't >> >> >> >> > > Hadoop >> >> >> >> > > > > > > >>> users/customers be better served if qualification eff= ort >> >> >> >> > > > > > > >>> went >> >> >> >> > > > into >> >> >> >> > > > > > > >>> Java7/8 versus Java6/7? >> >> >> >> > > > > > > >>> >> >> >> >> > > > > > > >>> Getting to Java7 as a development env (and Java8 as a >> >> >> >> > > > > > > >>> runtime >> >> >> >> > > > env) >> >> >> >> > > > > > > >>> seems like a no-brainer. Question is: How? >> >> >> >> > > > > > > >>> >> >> >> >> > > > > > > >>> On Tue, Apr 8, 2014 at 10:21 AM, Sandy Ryza < >> >> >> >> > > > > sandy.ryza@cloudera.com >> >> >> >> > > > > > > >> >> >> >> > > > > > > wrote: >> >> >> >> > > > > > > >>>> It might make sense to try to enumerate the benefits= of >> >> >> >> > > > switching >> >> >> >> > > > > to >> >> >> >> > > > > > > Java7 >> >> >> >> > > > > > > >>>> APIs and dependencies. IMO, the ones listed so far = on >> >> >> >> > > > > > > >>>> this >> >> >> >> > > > thread >> >> >> >> > > > > > > don't >> >> >> >> > > > > > > >>>> make a compelling enough case to drop Java6 in branc= h-2 >> >> >> >> > > > > > > >>>> on any >> >> >> >> > > > > time >> >> >> >> > > > > > > frame, >> >> >> >> > > > > > > >>>> even if this means supporting Java6 through 2015. F= or >> >> >> >> > > example, >> >> >> >> > > > > the >> >> >> >> > > > > > > change >> >> >> >> > > > > > > >>>> in RawLocalFileSystem semantics might be an >> incompatible >> >> >> >> > > change >> >> >> >> > > > > for >> >> >> >> > > > > > > >>>> branch-2 any way. >> >> >> >> > > > > > > >>>> >> >> >> >> > > > > > > >>>> >> >> >> >> > > > > > > >>>> On Tue, Apr 8, 2014 at 10:05 AM, Karthik Kambatla < >> >> >> >> > > > > > kasha@cloudera.com >> >> >> >> > > > > > > >wrote: >> >> >> >> > > > > > > >>>> >> >> >> >> > > > > > > >>>>> +1 to NOT breaking compatibility in branch-2. >> >> >> >> > > > > > > >>>>> >> >> >> >> > > > > > > >>>>> I think it is reasonable to require JDK7 for trunk,= if >> >> >> >> > > > > > > >>>>> we >> >> >> >> > > limit >> >> >> >> > > > > use >> >> >> >> > > > > > > of >> >> >> >> > > > > > > >>>>> JDK7-only API to security fixes etc. If we make oth= er >> >> >> >> > > > > optimizations >> >> >> >> > > > > > > (like >> >> >> >> > > > > > > >>>>> IO), it would be a pain to backport things to >> branch-2. >> >> >> >> > > > > > > >>>>> I >> >> >> >> > > guess >> >> >> >> > > > > > this >> >> >> >> > > > > > > all >> >> >> >> > > > > > > >>>>> depends on when we see ourselves shipping Hadoop-3. >> Any >> >> >> >> > > > > > > >>>>> ideas >> >> >> >> > > > on >> >> >> >> > > > > > > that? >> >> >> >> > > > > > > >>>>> >> >> >> >> > > > > > > >>>>> >> >> >> >> > > > > > > >>>>> On Tue, Apr 8, 2014 at 9:19 AM, Eli Collins < >> >> >> >> > > eli@cloudera.com> >> >> >> >> > > > > > > wrote: >> >> >> >> > > > > > > >>>>> >> >> >> >> > > > > > > >>>>> > On Tue, Apr 8, 2014 at 2:00 AM, Ottenheimer, Davi >> >> >> >> > > > > > > >>>>> > > >> Davi.Ottenheimer@emc.com>> wrote: >> >> >> >> > > > > > > >>>>> > >> From: Eli Collins [mailto:eli@cloudera.com] >> >> >> >> > > > > > > >>>>> > >> Sent: Monday, April 07, 2014 11:54 AM >> >> >> >> > > > > > > >>>>> > >> >> >> >> >> > > > > > > >>>>> > >> >> >> >> >> > > > > > > >>>>> > >> IMO we should not drop support for Java 6 in a >> >> >> >> > > > > > > >>>>> > >> minor >> >> >> >> > > > update >> >> >> >> > > > > > of a >> >> >> >> > > > > > > >>>>> stable >> >> >> >> > > > > > > >>>>> > >> release (v2). I don't think the larger Hadoop >> user >> >> >> >> > > > > > > >>>>> > >> base >> >> >> >> > > > > would >> >> >> >> > > > > > > find it >> >> >> >> > > > > > > >>>>> > >> acceptable that upgrading to a minor update >> caused >> >> >> >> > > > > > > >>>>> > >> their >> >> >> >> > > > > > > systems to >> >> >> >> > > > > > > >>>>> stop >> >> >> >> > > > > > > >>>>> > >> working because they didn't upgrade Java. Ther= e >> are >> >> >> >> > > people >> >> >> >> > > > > > still >> >> >> >> > > > > > > >>>>> getting >> >> >> >> > > > > > > >>>>> > >> support for Java 6. ... >> >> >> >> > > > > > > >>>>> > >> >> >> >> >> > > > > > > >>>>> > >> Thanks, >> >> >> >> > > > > > > >>>>> > >> Eli >> >> >> >> > > > > > > >>>>> > > >> >> >> >> > > > > > > >>>>> > > Hi Eli, >> >> >> >> > > > > > > >>>>> > > >> >> >> >> > > > > > > >>>>> > > Technically you are correct those with extended >> >> >> >> > > > > > > >>>>> > > support >> >> >> >> > > get >> >> >> >> > > > > > > critical >> >> >> >> > > > > > > >>>>> > security fixes for 6 until the end of 2016. I am >> >> >> >> > > > > > > >>>>> > curious >> >> >> >> > > > > whether >> >> >> >> > > > > > > many of >> >> >> >> > > > > > > >>>>> > those are in the Hadoop user base. Do you know? M= y >> >> >> >> > > > > > > >>>>> > guess is >> >> >> >> > > > the >> >> >> >> > > > > > > vast >> >> >> >> > > > > > > >>>>> > majority are within Oracle's official public end = of >> >> >> >> > > > > > > >>>>> > life, >> >> >> >> > > > which >> >> >> >> > > > > > > was over >> >> >> >> > > > > > > >>>>> 12 >> >> >> >> > > > > > > >>>>> > months ago. Even Premier support ended Dec 2013: >> >> >> >> > > > > > > >>>>> > > >> >> >> >> > > > > > > >>>>> > > http://www.oracle.com/technetwork/java/eol-< >> >> http://www.oracle.com/technetwork/java/eol-135779.ht> >> >> >> >> > 135779.ht >> >> >> >> > > > > > > >>>>> > > ml >> >> >> >> > > > > > > >>>>> > > >> >> >> >> > > > > > > >>>>> > > The end of Java 6 support carries much risk. It >> has >> >> >> >> > > > > > > >>>>> > > to be >> >> >> >> > > > > > > considered in >> >> >> >> > > > > > > >>>>> > terms of serious security vulnerabilities such as >> >> >> >> > > > CVE-2013-2465 >> >> >> >> > > > > > > with CVSS >> >> >> >> > > > > > > >>>>> > score 10.0. >> >> >> >> > > > > > > >>>>> > > >> >> >> >> > > > > > > >>>>> > > http://www.cvedetails.com/cve/CVE-2013-2465/ >> >> >> >> > > > > > > >>>>> > > >> >> >> >> > > > > > > >>>>> > > Since you mentioned "caused systems to stop" as= an >> >> >> >> > > example >> >> >> >> > > > of >> >> >> >> > > > > > > what >> >> >> >> > > > > > > >>>>> would >> >> >> >> > > > > > > >>>>> > be a concern to Hadoop users, please note the >> >> >> >> > > > > > > >>>>> > CVE-2013-2465 >> >> >> >> > > > > > > availability >> >> >> >> > > > > > > >>>>> > impact: >> >> >> >> > > > > > > >>>>> > > >> >> >> >> > > > > > > >>>>> > > "Complete (There is a total shutdown of the >> affected >> >> >> >> > > > > resource. >> >> >> >> > > > > > > The >> >> >> >> > > > > > > >>>>> > attacker can render the resource completely >> >> unavailable.)" >> >> >> >> > > > > > > >>>>> > > >> >> >> >> > > > > > > >>>>> > > This vulnerability was patched in Java 6 Update >> 51, >> >> >> >> > > > > > > >>>>> > > but >> >> >> >> > > > post >> >> >> >> > > > > > end >> >> >> >> > > > > > > of >> >> >> >> > > > > > > >>>>> > life. Apple pushed out the update specifically >> because >> >> >> >> > > > > > > >>>>> > of >> >> >> >> > > > this >> >> >> >> > > > > > > >>>>> > vulnerability (http://support.apple.com/kb/HT5717= ) >> as >> >> >> >> > > > > > > >>>>> > did >> >> >> >> > > > some >> >> >> >> > > > > > > other >> >> >> >> > > > > > > >>>>> > vendors privately, but for the majority of people >> >> >> >> > > > > > > >>>>> > using >> >> >> >> > > Java >> >> >> >> > > > 6 >> >> >> >> > > > > > > means they >> >> >> >> > > > > > > >>>>> > have a ticking time bomb. >> >> >> >> > > > > > > >>>>> > > >> >> >> >> > > > > > > >>>>> > > Allowing it to stay should be considered in ter= ms >> of >> >> >> >> > > > > accepting >> >> >> >> > > > > > > the >> >> >> >> > > > > > > >>>>> whole >> >> >> >> > > > > > > >>>>> > risk posture. >> >> >> >> > > > > > > >>>>> > > >> >> >> >> > > > > > > >>>>> > >> >> >> >> > > > > > > >>>>> > There are some who get extended support, but I >> suspect >> >> >> >> > > > > > > >>>>> > many >> >> >> >> > > > > just >> >> >> >> > > > > > > have >> >> >> >> > > > > > > >>>>> > a if-it's-not-broke mentality when it comes to >> >> >> >> > > > > > > >>>>> > production >> >> >> >> > > > > > > deployments. >> >> >> >> > > > > > > >>>>> > The current code supports both java6 and java7 an= d >> so >> >> >> >> > > allows >> >> >> >> > > > > > these >> >> >> >> > > > > > > >>>>> > people to remain compatible, while enabling other= s >> to >> >> >> >> > > upgrade >> >> >> >> > > > > to >> >> >> >> > > > > > > the >> >> >> >> > > > > > > >>>>> > java7 runtime. This seems like the right compromi= se >> >> >> >> > > > > > > >>>>> > for a >> >> >> >> > > > > stable >> >> >> >> > > > > > > >>>>> > release series. Again, absolutely makes sense for >> >> >> >> > > > > > > >>>>> > trunk (ie >> >> >> >> > > > v3) >> >> >> >> > > > > > to >> >> >> >> > > > > > > >>>>> > require java7 or greater. >> >> >> >> > > > > > > >>>>> > >> >> >> >> > > > > > > >>>>> >> >> >> >> > > > > > > >> >> >> >> >> > > > > > > >> >> >> >> >> > > > > > > >> >> >> >> >> > > > > > > >> -- >> >> >> >> > > > > > > >> - Tsuyoshi >> >> >> >> > > > > > > >> >> >> >> > > > > > >> >> >> >> > > > > > >> >> >> >> > > > > > >> >> >> >> > > > > > -- >> >> >> >> > > > > > Best regards, >> >> >> >> > > > > > >> >> >> >> > > > > > - Andy >> >> >> >> > > > > > >> >> >> >> > > > > > Problems worthy of attack prove their worth by hitting back= . - >> >> >> >> > > > > > Piet >> >> >> >> > > > Hein >> >> >> >> > > > > > (via Tom White) >> >> >> >> > > > > > >> >> >> >> > > > > >> >> >> >> > > > > -- >> >> >> >> > > > > CONFIDENTIALITY NOTICE >> >> >> >> > > > > NOTICE: This message is intended for the use of the individua= l >> or >> >> >> >> > > entity >> >> >> >> > > > to >> >> >> >> > > > > which it is addressed and may contain information that is >> >> >> >> > > > > confidential, privileged and exempt from disclosure under >> >> >> >> > > > > applicable law. If the >> >> >> >> > > reader >> >> >> >> > > > > of this message is not the intended recipient, you are hereby >> >> >> >> > > > > notified >> >> >> >> > > > that >> >> >> >> > > > > any printing, copying, dissemination, distribution, disclosur= e >> or >> >> >> >> > > > > forwarding of this communication is strictly prohibited. If y= ou >> >> >> >> > > > > have received this communication in error, please contact the >> >> >> >> > > > > sender >> >> >> >> > > > immediately >> >> >> >> > > > > and delete it from your system. Thank You. >> >> >> >> > > > > >> >> >> >> > > > >> >> >> >> > > >> >> >> >> > > -- >> >> >> >> > > CONFIDENTIALITY NOTICE >> >> >> >> > > NOTICE: This message is intended for the use of the individual or >> >> >> >> > > entity to which it is addressed and may contain information that = is >> >> >> >> > > confidential, privileged and exempt from disclosure under applica= ble >> >> >> >> > > law. If the reader of this message is not the intended recipient, >> you >> >> >> >> > > are hereby notified that any printing, copying, dissemination, >> >> >> >> > > distribution, disclosure or forwarding of this communication is >> >> >> >> > > strictly prohibited. If you have received this communication in >> error, >> >> >> >> > > please contact the sender immediately and delete it from your >> system. >> >> >> >> > Thank You. >> >> >> >> > > >> >> >> > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White)