hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Colin McCabe <cmcc...@alumni.cmu.edu>
Subject Re: Plans of moving towards JDK7 in trunk
Date Fri, 20 Jun 2014 18:02:36 GMT
Er, that should read "order in which it ran unit tests."

C.

On Fri, Jun 20, 2014 at 11:02 AM, Colin McCabe <cmccabe@alumni.cmu.edu> wrote:
> I think the important thing to do right now is to ensure our code
> works with jdk8.  This is similar to the work we did last year to fix
> issues that cropped up with jdk7.  There were just a few minor things
> like the error in which it ran unit tests that caused issues.
>
> I don't think it's out of the question to bump minVersion directly
> from 1.6 to 1.8.  Java 7 is EOL in less than a year, after all... at
> least according to the current roadmap here:
> http://www.oracle.com/technetwork/java/eol-135779.html
>
> We should try to do things in a way that causes the least disruption
> for users upgrading clusters, if possible...
>
> cheers,
> Colin
>
> On Thu, Jun 19, 2014 at 3:31 PM, Andrew Purtell <apurtell@apache.org> wrote:
>> There are a number of security (algorithm, not vulnerability) and
>> performance improvements that landed in 8, not 7. As a runtime for the
>> performance conscious, it might be recommendable. I've come across GC
>> issues in 6 or 7 where, talking with some Java platform folks, the first
>> suggested course of action is try again with 8. Would be be more of the
>> current moment if this discussion was about setting guidelines that
>> prescribe when and when not to use 8+ language features, or concurrency
>> library improvements that rely on intrinsics only available in the 8
>> runtime? Has the Java 6 ship sailed? Just set the minimum supported JDK and
>> runtime at 7 at next release? Use of the <> operator or multicatch wouldn't
>> and shouldn't need be debated, they are quite minor. On the other hand, I
>> would imagine discussion and debate on what 8+ language features might be
>> useful to use at some future time could be a lively one.
>>
>>
>>
>> On Wed, Jun 18, 2014 at 3:03 PM, Colin McCabe <cmccabe@alumni.cmu.edu>
>> wrote:
>>
>>> In CDH5, Cloudera encourages people to use JDK7.  JDK6 has been EOL
>>> for a while now and is not something we recommend.
>>>
>>> As we discussed before, everyone is in favor of upgrading to JDK7.
>>> Every cluster operator of a reasonably modern Hadoop should do it....
>>> whatever distro or release you run.  As developers, we run JDK7 as
>>> well.
>>>
>>> I'd just like to see a plan for when branch-2 (or some other branch)
>>> will create a stable release that drops support for JDK1.6.  If we
>>> don't have such a plan, I feel like it's too early to talk about this
>>> stuff.
>>>
>>> If we drop support for 1.6 in trunk but not in branch-2, we are
>>> fragmenting the project.  People will start writing unreleaseable code
>>> (because it doesn't work on branch-2) and we'll be back to the bad old
>>> days of Hadoop version fragmentation that branch-2 was intended to
>>> solve.  Backports will become harder.  The biggest problem is that
>>> trunk will start to depend on libraries or Maven plugins that branch-2
>>> can't even use, because they're JDK7+-only.
>>>
>>> Steve wrote: "if someone actually did file a bug on something on
>>> branch-2 which didn't work on Java 6 but went away on Java7+, we'd
>>> probably close it as a WORKSFORME".
>>>
>>> Steve, if this is true, we should just bump the minimum supported
>>> version for branch-2 to 1.7 today and resolve this.  If we truly
>>> believe that there are no issues here, then let's just decide to drop
>>> 1.6 in a specific future release of Hadoop 2.  If there are issues
>>> with releasing JDK1.7+ only code, then let's figure out what they are
>>> before proceeding.
>>>
>>> best,
>>> Colin
>>>
>>>
>>> On Wed, Jun 18, 2014 at 1:41 PM, Sandy Ryza <sandy.ryza@cloudera.com>
>>> wrote:
>>> > We do release warnings when we are aware of vulnerabilities in our
>>> > dependencies.
>>> >
>>> > However, unless I'm grossly misunderstanding, the vulnerability that you
>>> > point out is not a vulnerability within the context of our software.
>>> >  Hadoop doesn't try to sandbox within JVMs.  In a secure setup, any JVM
>>> > running non-trusted user code is running as that user, so "breaking out"
>>> > doesn't offer the ability to do anything malicious.
>>> >
>>> > -Sandy
>>> >
>>> > On Wed, Jun 18, 2014 at 1:30 PM, Ottenheimer, Davi <
>>> Davi.Ottenheimer@emc.com
>>> >> wrote:
>>> >
>>> >> Andrew,
>>> >>
>>> >>
>>> >>
>>> >> “I don't see any point to switching” is an interesting perspective,
>>> given
>>> >> the well-known risks of running unsafe software. Clearly customer best
>>> >> interest is stability. JDK6 is in a known unsafe state. The longer
>>> anyone
>>> >> delays the necessary transition to safety the longer the door is left
>>> open
>>> >> to predictable disaster.
>>> >>
>>> >>
>>> >>
>>> >> You also said "we still test and support JDK6". I searched but have not
>>> >> been able to find Cloudera critical security fixes for JDK6.
>>> >>
>>> >>
>>> >>
>>> >> Can you clarify, for example, Java 6 Update 51 for CVE-2013-2465? In
>>> other
>>> >> words, did you release to your customers any kind of public alert or
>>> >> warning of this CVSS 10.0 event as part of your JDK6 support?
>>> >>
>>> >>
>>> >>
>>> >> http://www.cvedetails.com/cve/CVE-2013-2465/
>>> >>
>>> >>
>>> >>
>>> >> If you are not releasing your own security fixes for JDK6 post-EOL would
>>> >> it perhaps be safer to say Cloudera is hands-off; neither supports, nor
>>> >> opposes the known insecure and deprecated/unpatched JDK?
>>> >>
>>> >>
>>> >>
>>> >> I mentioned before in this thread the Oracle support timeline:
>>> >>
>>> >>
>>> >>
>>> >> - official public EOL (end of life) was more than a year ago
>>> >>
>>> >> - premier support ended more than six months ago
>>> >>
>>> >> - extended support may get critical security fixes until the end of 2016
>>> >>
>>> >>
>>> >>
>>> >> Given this timeline, does Cloudera officially take responsibility for
>>> >> Hadoop customer safety? Are you going to be releasing critical security
>>> >> fixes to a known unsafe JDK?
>>> >>
>>> >>
>>> >>
>>> >> Davi
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> > -----Original Message-----
>>> >>
>>> >> > From: Andrew Wang [mailto:andrew.wang@cloudera.com]
>>> >>
>>> >> > Sent: Wednesday, June 18, 2014 12:33 PM
>>> >>
>>> >> > To: common-dev@hadoop.apache.org
>>> >>
>>> >> > Subject: Re: Plans of moving towards JDK7 in trunk
>>> >>
>>> >> >
>>> >>
>>> >> > Actually, a lot of our customers are still on JDK6, so if anything,
>>> its
>>> >> popularity
>>> >>
>>> >> > hasn't significantly decreased. We still test and support JDK6 for
>>> CDH4
>>> >> and
>>> >>
>>> >> > CDH5. The claim that branch-2 is effectively JDK7 because no one
>>> supports
>>> >>
>>> >> > JDK6 is untrue.
>>> >>
>>> >> >
>>> >>
>>> >> > One issue with your proposal is that java 7+ libraries can have
>>> >> incompatible
>>> >>
>>> >> > APIs compared to their java 6 versions. Guava moves very quickly with
>>> >> regard
>>> >>
>>> >> > to the deprecate+remove cycle. This means branch-2 and trunk
>>> divergence,
>>> >>
>>> >> > as we're stuck using different Guava APIs to do the same thing.
>>> >>
>>> >> >
>>> >>
>>> >> > No one's arguing against moving to Java 7+ in trunk eventually, but
>>> >> there isn't
>>> >>
>>> >> > a clear plan for a trunk-based release. I don't see any point to
>>> >> switching trunk
>>> >>
>>> >> > over until that's true, for the aforementioned reasons.
>>> >>
>>> >> >
>>> >>
>>> >> > Best,
>>> >>
>>> >> > Andrew
>>> >>
>>> >> >
>>> >>
>>> >> >
>>> >>
>>> >> > On Wed, Jun 18, 2014 at 12:08 PM, Steve Loughran
>>> >>
>>> >> > <stevel@hortonworks.com<mailto:stevel@hortonworks.com>>
>>> >>
>>> >> > wrote:
>>> >>
>>> >> >
>>> >>
>>> >> > > I also think we need to recognise that its been three months since
>>> >>
>>> >> > > that last discussion, and Java 6 has not suddenly burst back into
>>> >>
>>> >> > > popularity
>>> >>
>>> >> > >
>>> >>
>>> >> > >
>>> >>
>>> >> > >    - nobody providing commercial support for Hadoop is offering
>>> >> branch-2
>>> >>
>>> >> > >    support on Java 6 AFAIK
>>> >>
>>> >> > >    - therefore, nobody is testing it at scale except privately, and
>>> >> they
>>> >>
>>> >> > >    aren't reporting bugs if they are
>>> >>
>>> >> > >    - if someone actually did file a bug on something on branch-2
>>> which
>>> >>
>>> >> > >    didn't work on Java 6 but went away on Java7+, we'd probably
>>> close
>>> >>
>>> >> > > it as a
>>> >>
>>> >> > >    WORKSFORME
>>> >>
>>> >> > >
>>> >>
>>> >> > >
>>> >>
>>> >> > > whether we acknowledge it or not, Hadoop 2.x is now really Java 7+.
>>> >>
>>> >> > >
>>> >>
>>> >> > > We do all agree that hadoop 3 will not be java 6, so the only issue
>>> is
>>> >>
>>> >> > > "when and how to make that transition".
>>> >>
>>> >> > >
>>> >>
>>> >> > > That patch of mine just makes it possible to do today.
>>> >>
>>> >> > >
>>> >>
>>> >> > > I have actually jumped to Java7 in the slider project, and actually
>>> >>
>>> >> > > being using Java 8 and twill; the new language features there are
>>> >>
>>> >> > > significant and would be great to use in Hadoop *at some point in
>>> the
>>> >>
>>> >> > > future*
>>> >>
>>> >> > >
>>> >>
>>> >> > > For Java 7 though, based on that experience, the language changes
>>> are
>>> >>
>>> >> > > convenient but not essential
>>> >>
>>> >> > >
>>> >>
>>> >> > >    - try-with-resources simply swallows close failures without the
>>> log
>>> >>
>>> >> > >    integration we have with IOUtils.closeStream(), so shoudn't be
>>> used
>>> >> in
>>> >>
>>> >> > >    hadoop core anyway.
>>> >>
>>> >> > >    - string based switching: convenient, but not critical
>>> >>
>>> >> > >    - type inference on template constructors. Modern IDEs handle the
>>> >> pain
>>> >>
>>> >> > >    anyway
>>> >>
>>> >> > >
>>> >>
>>> >> > > The only feature I like is multi-catch and typed rethrow
>>> >>
>>> >> > >
>>> >>
>>> >> > > catch(IOException | ExitException e) {  log.warn(e.toString();
>>>  throw
>>> >>
>>> >> > > e; }
>>> >>
>>> >> > >
>>> >>
>>> >> > > this would make "e" look like Exception, but when rethrown go back
>>> to
>>> >>
>>> >> > > its original type.
>>> >>
>>> >> > >
>>> >>
>>> >> > > This reduces duplicate work, and is the bit l actually value. Is it
>>> >>
>>> >> > > enough to justify making code incompatible across branches? No.
>>> >>
>>> >> > >
>>> >>
>>> >> > > So i'm going to propose this, and would like to start a vote on it
>>> >>
>>> >> > > soon
>>> >>
>>> >> > >
>>> >>
>>> >> > >
>>> >>
>>> >> > >    1. we parameterize java versions in the POMs on all branches,
>>> with
>>> >>
>>> >> > >    separate JDK versions and Java language
>>> >>
>>> >> > >    2. branch-2: java-6-language and JDK-6 minimum JDK
>>> >>
>>> >> > >    3. trunk: java-6-language and JDK-7 minimum JDK
>>> >>
>>> >> > >
>>> >>
>>> >> > > This would guarantee that none of the java 7 language features went
>>> >>
>>> >> > > in, but we could move trunk up to java 7+ only libraries (jersey,
>>> >>
>>> >> > > guava). Adopting
>>> >>
>>> >> > > JDK7 features then becomes no more different from adopting java7+
>>> >>
>>> >> > > libraries: those bits of code that have moved can't be backported.
>>> >>
>>> >> > >
>>> >>
>>> >> > > -Steve
>>> >>
>>> >> > >
>>> >>
>>> >> > >
>>> >>
>>> >> > >
>>> >>
>>> >> > >
>>> >>
>>> >> > >
>>> >>
>>> >> > > On 17 June 2014 22:08, Andrew Wang <andrew.wang@cloudera.com
>>> <mailto:
>>> >> andrew.wang@cloudera.com>>
>>> >>
>>> >> > wrote:
>>> >>
>>> >> > >
>>> >>
>>> >> > > > Reviving this thread, I noticed there's been a patch and +1 on
>>> >>
>>> >> > > > HADOOP-10530, and I don't think we actually reached a conclusion.
>>> >>
>>> >> > > >
>>> >>
>>> >> > > > I (and others) have expressed concerns about moving to JDK7 for
>>> >> trunk.
>>> >>
>>> >> > > > Summarizing a few points:
>>> >>
>>> >> > > >
>>> >>
>>> >> > > > - We can't move to JDK7 in branch-2 because of compatibility
>>> >>
>>> >> > > > - branch-2 is currently the only Hadoop release vehicle, there are
>>> >>
>>> >> > > > no
>>> >>
>>> >> > > plans
>>> >>
>>> >> > > > for a trunk-based Hadoop 3
>>> >>
>>> >> > > > - Introducing JDK7-only APIs in trunk will increase divergence
>>> with
>>> >>
>>> >> > > > branch-2 and make backports harder
>>> >>
>>> >> > > > - Almost all developers care only about branch-2, since it is the
>>> >>
>>> >> > > > only release vehicle
>>> >>
>>> >> > > >
>>> >>
>>> >> > > > With this in mind, I struggle to see any upsides to introducing
>>> >>
>>> >> > > > JDK7-only APIs to trunk. Please let's not do anything on
>>> >>
>>> >> > > > HADOOP-10530 or related until we agree on this.
>>> >>
>>> >> > > >
>>> >>
>>> >> > > > Thanks,
>>> >>
>>> >> > > > Andrew
>>> >>
>>> >> > > >
>>> >>
>>> >> > > >
>>> >>
>>> >> > > > On Mon, Apr 14, 2014 at 3:31 PM, Steve Loughran
>>> >>
>>> >> > > > <stevel@hortonworks.com<mailto:stevel@hortonworks.com>>
>>> >>
>>> >> > > > wrote:
>>> >>
>>> >> > > >
>>> >>
>>> >> > > > > On 14 April 2014 17:46, Andrew Purtell <apurtell@apache.org
>>> >> <mailto:apurtell@apache.org>> wrote:
>>> >>
>>> >> > > > >
>>> >>
>>> >> > > > > > How well is trunk tested? Does anyone deploy it with real
>>> >>
>>> >> > > applications
>>> >>
>>> >> > > > > > running on top? When will the trunk codebase next be the basis
>>> >>
>>> >> > > > > > for a production release? An impromptu diff of hadoop-common
>>> >>
>>> >> > > > > > trunk against
>>> >>
>>> >> > > > > > branch-2 as of today is 38,625 lines. Can they be said to be
>>> the
>>> >>
>>> >> > > > > > same animal? I ask because any disincentive toward putting
>>> code
>>> >>
>>> >> > > > > > in trunk
>>> >>
>>> >> > > is
>>> >>
>>> >> > > > > > beside the point, if the only target worth pursuing today is
>>> >>
>>> >> > > > > > branch-2 unless one doesn't care if the code is released for
>>> >>
>>> >> > production use.
>>> >>
>>> >> > > > > > Questions on whither JDK6 or JDK7+ (or JRE6 versus JRE7+) only
>>> >>
>>> >> > > > > > matter
>>> >>
>>> >> > > > for
>>> >>
>>> >> > > > > > the vast majority of Hadoopers if talking about branch-2.
>>> >>
>>> >> > > > > >
>>> >>
>>> >> > > > > >
>>> >>
>>> >> > > > > I think its partly a timescale issue; its also because the 1-2
>>> >>
>>> >> > > transition
>>> >>
>>> >> > > > > was so significant, especially at the YARN layer, that it's
>>> still
>>> >>
>>> >> > > taking
>>> >>
>>> >> > > > > time to trickle through.
>>> >>
>>> >> > > > >
>>> >>
>>> >> > > > > If you do want code to ship this year, branch-2 is where you are
>>> >>
>>> >> > > > > going
>>> >>
>>> >> > > to
>>> >>
>>> >> > > > > try and get it in -and like you say, that's where things get
>>> tried
>>> >>
>>> >> > > > > in
>>> >>
>>> >> > > the
>>> >>
>>> >> > > > > field. At the same time, the constraints of stability are
>>> holding
>>> >>
>>> >> > > > > us
>>> >>
>>> >> > > back
>>> >>
>>> >> > > > > -already-.
>>> >>
>>> >> > > > >
>>> >>
>>> >> > > > > I don't see why we should have such another major 1-2 transition
>>> >>
>>> >> > > > > in
>>> >>
>>> >> > > > future;
>>> >>
>>> >> > > > > the rate that Arun is pushing out 2.x releases its almost back
>>> to
>>> >>
>>> >> > > > > the
>>> >>
>>> >> > > > 0.1x
>>> >>
>>> >> > > > > timescale -though at that point most people were fending for
>>> >>
>>> >> > > > > themselves
>>> >>
>>> >> > > > and
>>> >>
>>> >> > > > > expectations of stability were less. We do want smaller version
>>> >>
>>> >> > > > increments
>>> >>
>>> >> > > > > in future, which branch-2 is -mostly- delivering.
>>> >>
>>> >> > > > >
>>> >>
>>> >> > > > > While Java 7 doesn't have some must-have features, Java 8 is a
>>> >>
>>> >> > > > significant
>>> >>
>>> >> > > > > improvement in the language, and we should be looking ahead to
>>> >>
>>> >> > > > > that,
>>> >>
>>> >> > > > maybe
>>> >>
>>> >> > > > > even doing some leading-edge work on the side, so the same
>>> >>
>>> >> > > > > discussion doesn't come up in two years time when java 7 goes
>>> EOL.
>>> >>
>>> >> > > > >
>>> >>
>>> >> > > > >
>>> >>
>>> >> > > > > -steve
>>> >>
>>> >> > > > >
>>> >>
>>> >> > > > > (personal opinions only, etc, )
>>> >>
>>> >> > > > >
>>> >>
>>> >> > > > >
>>> >>
>>> >> > > > > >
>>> >>
>>> >> > > > > > On Mon, Apr 14, 2014 at 9:22 AM, Colin McCabe <
>>> >>
>>> >> > > cmccabe@alumni.cmu.edu<mailto:cmccabe@alumni.cmu.edu>
>>> >>
>>> >> > > > > > >wrote:
>>> >>
>>> >> > > > > >
>>> >>
>>> >> > > > > > > I think the bottom line here is that as long as our stable
>>> >>
>>> >> > > > > > > release uses JDK6, there is going to be a very, very strong
>>> >>
>>> >> > > > > > > disincentive to put any code which can't run on JDK6 into
>>> >> trunk.
>>> >>
>>> >> > > > > > >
>>> >>
>>> >> > > > > > > Like I said earlier, the traditional reason for putting
>>> >>
>>> >> > > > > > > something
>>> >>
>>> >> > > in
>>> >>
>>> >> > > > > > > trunk but not the stable release is that it needs more
>>> testing.
>>> >>
>>> >> > >  If a
>>> >>
>>> >> > > > > > > stable release that drops support for JDK6 is more than a
>>> year
>>> >>
>>> >> > > away,
>>> >>
>>> >> > > > > > > does it make sense to put anything in trunk like that?  What
>>> >>
>>> >> > > > > > > might need more than a year of testing?  Certainly not
>>> changes
>>> >>
>>> >> > > > > > > to LocalFileSystem to use the new APIs.  I also don't think
>>> an
>>> >>
>>> >> > > > > > > upgrade
>>> >>
>>> >> > > > to
>>> >>
>>> >> > > > > > > various libraries qualifies.
>>> >>
>>> >> > > > > > >
>>> >>
>>> >> > > > > > > It might be best to shelve this for now, like we've done in
>>> >>
>>> >> > > > > > > the
>>> >>
>>> >> > > past,
>>> >>
>>> >> > > > > > > until we're ready to talk about a stable release that
>>> requires
>>> >>
>>> >> > > JDK7+.
>>> >>
>>> >> > > > > > > At least that's my feeling.
>>> >>
>>> >> > > > > > >
>>> >>
>>> >> > > > > > > If we're really desperate for the new file APIs JDK7
>>> provides,
>>> >>
>>> >> > > > > > > we could consider using loadable modules for it in branch-2.
>>> >>
>>> >> > > > > > > This is similar to how we provide JNI versions of certain
>>> >>
>>> >> > > > > > > things on certain platforms, without dropping support for
>>> the
>>> >> other
>>> >>
>>> >> > platforms.
>>> >>
>>> >> > > > > > >
>>> >>
>>> >> > > > > > > best,
>>> >>
>>> >> > > > > > > Colin
>>> >>
>>> >> > > > > > >
>>> >>
>>> >> > > > > > > On Sun, Apr 13, 2014 at 10:39 AM, Raymie Stata <
>>> >>
>>> >> > > rstata@altiscale.com<mailto:rstata@altiscale.com>
>>> >>
>>> >> > > > >
>>> >>
>>> >> > > > > > > wrote:
>>> >>
>>> >> > > > > > > > There's an outstanding question addressed to me: "Are
>>> there
>>> >>
>>> >> > > > > particular
>>> >>
>>> >> > > > > > > > features or new dependencies that you would like to
>>> >>
>>> >> > > > > > > > contribute
>>> >>
>>> >> > > (or
>>> >>
>>> >> > > > > see
>>> >>
>>> >> > > > > > > > contributed) that require using the Java 1.7 APIs?"  The
>>> >>
>>> >> > > > > > > > question misses the point: We'd figure out how to write
>>> >>
>>> >> > > > > > > > something we
>>> >>
>>> >> > > wanted
>>> >>
>>> >> > > > to
>>> >>
>>> >> > > > > > > > contribute to Hadoop against the APIs of Java4 if that's
>>> >>
>>> >> > > > > > > > what it
>>> >>
>>> >> > > > took
>>> >>
>>> >> > > > > > > > to get them into a stable release.  And at current course
>>> >>
>>> >> > > > > > > > and
>>> >>
>>> >> > > > speed,
>>> >>
>>> >> > > > > > > > that's how ridiculous things could get.
>>> >>
>>> >> > > > > > > >
>>> >>
>>> >> > > > > > > > To summarize, it seems like there's a vague consensus that
>>> >>
>>> >> > > > > > > > it
>>> >>
>>> >> > > might
>>> >>
>>> >> > > > > be
>>> >>
>>> >> > > > > > > > okay to eventually allow the use of Java7 in trunk, but
>>> >>
>>> >> > > > > > > > there's
>>> >>
>>> >> > > no
>>> >>
>>> >> > > > > > > > decision.  And there's been no answer to the concern that
>>> >>
>>> >> > > > > > > > even if
>>> >>
>>> >> > > > > such
>>> >>
>>> >> > > > > > > > dependencies were allowed in Java7, the only people using
>>> >>
>>> >> > > > > > > > them
>>> >>
>>> >> > > > would
>>> >>
>>> >> > > > > > > > be people who uninterested in getting their patches into a
>>> >>
>>> >> > > > > > > > stable release of Hadoop on any knowable timeframe, which
>>> >>
>>> >> > > > > > > > doesn't bode
>>> >>
>>> >> > > > well
>>> >>
>>> >> > > > > > > > for the ability to stabilize that Java7 code when it comes
>>> >>
>>> >> > > > > > > > time
>>> >>
>>> >> > > to
>>> >>
>>> >> > > > > > > > attempt to.
>>> >>
>>> >> > > > > > > >
>>> >>
>>> >> > > > > > > > I don't have more to add, so I'll go back to lurking.
>>>  It'll
>>> >>
>>> >> > > > > > > > be interesting to see where we'll be standing a year from
>>> >> now.
>>> >>
>>> >> > > > > > > >
>>> >>
>>> >> > > > > > > > On Sun, Apr 13, 2014 at 2:09 AM, Tsuyoshi OZAWA
>>> >>
>>> >> > > > > > > > <ozawa.tsuyoshi@gmail.com<mailto:ozawa.tsuyoshi@gmail.com
>>> >>
>>> >> wrote:
>>> >>
>>> >> > > > > > > >> Hi,
>>> >>
>>> >> > > > > > > >>
>>> >>
>>> >> > > > > > > >> +1 for Karthik's idea(non-binding).
>>> >>
>>> >> > > > > > > >>
>>> >>
>>> >> > > > > > > >> IMO, we should keep the compatibility between JDK 6 and
>>> JDK
>>> >>
>>> >> > > > > > > >> 7 on
>>> >>
>>> >> > > > > both
>>> >>
>>> >> > > > > > > branch-1
>>> >>
>>> >> > > > > > > >> and branch-2, because users can be using them. For future
>>> >>
>>> >> > > releases
>>> >>
>>> >> > > > > > that
>>> >>
>>> >> > > > > > > we can
>>> >>
>>> >> > > > > > > >> declare breaking compatibility(e.g. 3.0.0 release), we
>>> can
>>> >>
>>> >> > > > > > > >> use
>>> >>
>>> >> > > > JDK 7
>>> >>
>>> >> > > > > > > >> features if we
>>> >>
>>> >> > > > > > > >> can get benefits. However, it can increase maintenance
>>> >>
>>> >> > > > > > > >> costs and
>>> >>
>>> >> > > > > > > distributes the
>>> >>
>>> >> > > > > > > >> efforts of contributions to maintain branches. Then, I
>>> >>
>>> >> > > > > > > >> think it
>>> >>
>>> >> > > is
>>> >>
>>> >> > > > > > > >> reasonable approach
>>> >>
>>> >> > > > > > > >> that we use limited and minimum JDK-7 APIs when we have
>>> >>
>>> >> > > > > > > >> reasons
>>> >>
>>> >> > > we
>>> >>
>>> >> > > > > > need
>>> >>
>>> >> > > > > > > to use
>>> >>
>>> >> > > > > > > >> the features.
>>> >>
>>> >> > > > > > > >> By the way, if we start to use JDK 7 APIs, we should
>>> >>
>>> >> > > > > > > >> declare the
>>> >>
>>> >> > > > > basis
>>> >>
>>> >> > > > > > > >> when to use
>>> >>
>>> >> > > > > > > >> JDK 7 APIs on Wiki not to confuse contributors.
>>> >>
>>> >> > > > > > > >>
>>> >>
>>> >> > > > > > > >> Thanks,
>>> >>
>>> >> > > > > > > >> - Tsuyoshi
>>> >>
>>> >> > > > > > > >>
>>> >>
>>> >> > > > > > > >> On Wed, Apr 9, 2014 at 11:44 AM, Raymie Stata <
>>> >>
>>> >> > > > rstata@altiscale.com<mailto:rstata@altiscale.com>
>>> >>
>>> >> > > > > >
>>> >>
>>> >> > > > > > > wrote:
>>> >>
>>> >> > > > > > > >>>> It might make sense to try to enumerate the benefits of
>>> >>
>>> >> > > > switching
>>> >>
>>> >> > > > > to
>>> >>
>>> >> > > > > > > >>>> Java7 APIs and dependencies.
>>> >>
>>> >> > > > > > > >>>
>>> >>
>>> >> > > > > > > >>>   - Java7 introduced a huge number of language,
>>> byte-code,
>>> >>
>>> >> > > > > > > >>> API,
>>> >>
>>> >> > > > and
>>> >>
>>> >> > > > > > > >>> tooling enhancements!  Just to name a few:
>>> >>
>>> >> > > > > > > >>> try-with-resources,
>>> >>
>>> >> > > > > newer
>>> >>
>>> >> > > > > > > >>> and stronger encyrption methods, more scalable
>>> concurrency
>>> >>
>>> >> > > > > > primitives.
>>> >>
>>> >> > > > > > > >>>  See
>>> >>
>>> >> > > > > > > >>>
>>> http://www.slideshare.net/boulderjug/55-things-in-java-7
>>> >>
>>> >> > > > > > > >>>
>>> >>
>>> >> > > > > > > >>>   - We can't update current dependencies, and we can't
>>> add
>>> >>
>>> >> > > > > > > >>> cool
>>> >>
>>> >> > > > new
>>> >>
>>> >> > > > > > > ones.
>>> >>
>>> >> > > > > > > >>>
>>> >>
>>> >> > > > > > > >>>   - Putting language/APIs aside, don't forget that a
>>> huge
>>> >>
>>> >> > > amount
>>> >>
>>> >> > > > of
>>> >>
>>> >> > > > > > > effort
>>> >>
>>> >> > > > > > > >>> goes into qualifying for Java6 (at least, I hope the
>>> folks
>>> >>
>>> >> > > > claiming
>>> >>
>>> >> > > > > > to
>>> >>
>>> >> > > > > > > >>> support Java6 are putting in such an effort :-).
>>>  Wouldn't
>>> >>
>>> >> > > Hadoop
>>> >>
>>> >> > > > > > > >>> users/customers be better served if qualification effort
>>> >>
>>> >> > > > > > > >>> went
>>> >>
>>> >> > > > into
>>> >>
>>> >> > > > > > > >>> Java7/8 versus Java6/7?
>>> >>
>>> >> > > > > > > >>>
>>> >>
>>> >> > > > > > > >>> Getting to Java7 as a development env (and Java8 as a
>>> >>
>>> >> > > > > > > >>> runtime
>>> >>
>>> >> > > > env)
>>> >>
>>> >> > > > > > > >>> seems like a no-brainer.  Question is: How?
>>> >>
>>> >> > > > > > > >>>
>>> >>
>>> >> > > > > > > >>> On Tue, Apr 8, 2014 at 10:21 AM, Sandy Ryza <
>>> >>
>>> >> > > > > sandy.ryza@cloudera.com<mailto:sandy.ryza@cloudera.com>
>>> >>
>>> >> > > > > > >
>>> >>
>>> >> > > > > > > wrote:
>>> >>
>>> >> > > > > > > >>>> It might make sense to try to enumerate the benefits of
>>> >>
>>> >> > > > switching
>>> >>
>>> >> > > > > to
>>> >>
>>> >> > > > > > > Java7
>>> >>
>>> >> > > > > > > >>>> APIs and dependencies.  IMO, the ones listed so far on
>>> >>
>>> >> > > > > > > >>>> this
>>> >>
>>> >> > > > thread
>>> >>
>>> >> > > > > > > don't
>>> >>
>>> >> > > > > > > >>>> make a compelling enough case to drop Java6 in branch-2
>>> >>
>>> >> > > > > > > >>>> on any
>>> >>
>>> >> > > > > time
>>> >>
>>> >> > > > > > > frame,
>>> >>
>>> >> > > > > > > >>>> even if this means supporting Java6 through 2015.  For
>>> >>
>>> >> > > example,
>>> >>
>>> >> > > > > the
>>> >>
>>> >> > > > > > > change
>>> >>
>>> >> > > > > > > >>>> in RawLocalFileSystem semantics might be an
>>> incompatible
>>> >>
>>> >> > > change
>>> >>
>>> >> > > > > for
>>> >>
>>> >> > > > > > > >>>> branch-2 any way.
>>> >>
>>> >> > > > > > > >>>>
>>> >>
>>> >> > > > > > > >>>>
>>> >>
>>> >> > > > > > > >>>> On Tue, Apr 8, 2014 at 10:05 AM, Karthik Kambatla <
>>> >>
>>> >> > > > > > kasha@cloudera.com<mailto:kasha@cloudera.com>
>>> >>
>>> >> > > > > > > >wrote:
>>> >>
>>> >> > > > > > > >>>>
>>> >>
>>> >> > > > > > > >>>>> +1 to NOT breaking compatibility in branch-2.
>>> >>
>>> >> > > > > > > >>>>>
>>> >>
>>> >> > > > > > > >>>>> I think it is reasonable to require JDK7 for trunk, if
>>> >>
>>> >> > > > > > > >>>>> we
>>> >>
>>> >> > > limit
>>> >>
>>> >> > > > > use
>>> >>
>>> >> > > > > > > of
>>> >>
>>> >> > > > > > > >>>>> JDK7-only API to security fixes etc. If we make other
>>> >>
>>> >> > > > > optimizations
>>> >>
>>> >> > > > > > > (like
>>> >>
>>> >> > > > > > > >>>>> IO), it would be a pain to backport things to
>>> branch-2.
>>> >>
>>> >> > > > > > > >>>>> I
>>> >>
>>> >> > > guess
>>> >>
>>> >> > > > > > this
>>> >>
>>> >> > > > > > > all
>>> >>
>>> >> > > > > > > >>>>> depends on when we see ourselves shipping Hadoop-3.
>>> Any
>>> >>
>>> >> > > > > > > >>>>> ideas
>>> >>
>>> >> > > > on
>>> >>
>>> >> > > > > > > that?
>>> >>
>>> >> > > > > > > >>>>>
>>> >>
>>> >> > > > > > > >>>>>
>>> >>
>>> >> > > > > > > >>>>> On Tue, Apr 8, 2014 at 9:19 AM, Eli Collins <
>>> >>
>>> >> > > eli@cloudera.com<mailto:eli@cloudera.com>>
>>> >>
>>> >> > > > > > > wrote:
>>> >>
>>> >> > > > > > > >>>>>
>>> >>
>>> >> > > > > > > >>>>> > On Tue, Apr 8, 2014 at 2:00 AM, Ottenheimer, Davi
>>> >>
>>> >> > > > > > > >>>>> > <Davi.Ottenheimer@emc.com<mailto:
>>> >> Davi.Ottenheimer@emc.com>> wrote:
>>> >>
>>> >> > > > > > > >>>>> > >> From: Eli Collins [mailto:eli@cloudera.com]
>>> >>
>>> >> > > > > > > >>>>> > >> Sent: Monday, April 07, 2014 11:54 AM
>>> >>
>>> >> > > > > > > >>>>> > >>
>>> >>
>>> >> > > > > > > >>>>> > >>
>>> >>
>>> >> > > > > > > >>>>> > >> IMO we should not drop support for Java 6 in a
>>> >>
>>> >> > > > > > > >>>>> > >> minor
>>> >>
>>> >> > > > update
>>> >>
>>> >> > > > > > of a
>>> >>
>>> >> > > > > > > >>>>> stable
>>> >>
>>> >> > > > > > > >>>>> > >> release (v2).  I don't think the larger Hadoop
>>> user
>>> >>
>>> >> > > > > > > >>>>> > >> base
>>> >>
>>> >> > > > > would
>>> >>
>>> >> > > > > > > find it
>>> >>
>>> >> > > > > > > >>>>> > >> acceptable that upgrading to a minor update
>>> caused
>>> >>
>>> >> > > > > > > >>>>> > >> their
>>> >>
>>> >> > > > > > > systems to
>>> >>
>>> >> > > > > > > >>>>> stop
>>> >>
>>> >> > > > > > > >>>>> > >> working because they didn't upgrade Java. There
>>> are
>>> >>
>>> >> > > people
>>> >>
>>> >> > > > > > still
>>> >>
>>> >> > > > > > > >>>>> getting
>>> >>
>>> >> > > > > > > >>>>> > >> support for Java 6. ...
>>> >>
>>> >> > > > > > > >>>>> > >>
>>> >>
>>> >> > > > > > > >>>>> > >> Thanks,
>>> >>
>>> >> > > > > > > >>>>> > >> Eli
>>> >>
>>> >> > > > > > > >>>>> > >
>>> >>
>>> >> > > > > > > >>>>> > > Hi Eli,
>>> >>
>>> >> > > > > > > >>>>> > >
>>> >>
>>> >> > > > > > > >>>>> > > Technically you are correct those with extended
>>> >>
>>> >> > > > > > > >>>>> > > support
>>> >>
>>> >> > > get
>>> >>
>>> >> > > > > > > critical
>>> >>
>>> >> > > > > > > >>>>> > security fixes for 6 until the end of 2016. I am
>>> >>
>>> >> > > > > > > >>>>> > curious
>>> >>
>>> >> > > > > whether
>>> >>
>>> >> > > > > > > many of
>>> >>
>>> >> > > > > > > >>>>> > those are in the Hadoop user base. Do you know? My
>>> >>
>>> >> > > > > > > >>>>> > guess is
>>> >>
>>> >> > > > the
>>> >>
>>> >> > > > > > > vast
>>> >>
>>> >> > > > > > > >>>>> > majority are within Oracle's official public end of
>>> >>
>>> >> > > > > > > >>>>> > life,
>>> >>
>>> >> > > > which
>>> >>
>>> >> > > > > > > was over
>>> >>
>>> >> > > > > > > >>>>> 12
>>> >>
>>> >> > > > > > > >>>>> > months ago. Even Premier support ended Dec 2013:
>>> >>
>>> >> > > > > > > >>>>> > >
>>> >>
>>> >> > > > > > > >>>>> > > http://www.oracle.com/technetwork/java/eol-<
>>> >> http://www.oracle.com/technetwork/java/eol-135779.ht>
>>> >>
>>> >> > 135779.ht<http://www.oracle.com/technetwork/java/eol-135779.ht>
>>> >>
>>> >> > > > > > > >>>>> > > ml
>>> >>
>>> >> > > > > > > >>>>> > >
>>> >>
>>> >> > > > > > > >>>>> > > The end of Java 6 support carries much risk. It
>>> has
>>> >>
>>> >> > > > > > > >>>>> > > to be
>>> >>
>>> >> > > > > > > considered in
>>> >>
>>> >> > > > > > > >>>>> > terms of serious security vulnerabilities such as
>>> >>
>>> >> > > > CVE-2013-2465
>>> >>
>>> >> > > > > > > with CVSS
>>> >>
>>> >> > > > > > > >>>>> > score 10.0.
>>> >>
>>> >> > > > > > > >>>>> > >
>>> >>
>>> >> > > > > > > >>>>> > > http://www.cvedetails.com/cve/CVE-2013-2465/
>>> >>
>>> >> > > > > > > >>>>> > >
>>> >>
>>> >> > > > > > > >>>>> > > Since you mentioned "caused systems to stop" as an
>>> >>
>>> >> > > example
>>> >>
>>> >> > > > of
>>> >>
>>> >> > > > > > > what
>>> >>
>>> >> > > > > > > >>>>> would
>>> >>
>>> >> > > > > > > >>>>> > be a concern to Hadoop users, please note the
>>> >>
>>> >> > > > > > > >>>>> > CVE-2013-2465
>>> >>
>>> >> > > > > > > availability
>>> >>
>>> >> > > > > > > >>>>> > impact:
>>> >>
>>> >> > > > > > > >>>>> > >
>>> >>
>>> >> > > > > > > >>>>> > > "Complete (There is a total shutdown of the
>>> affected
>>> >>
>>> >> > > > > resource.
>>> >>
>>> >> > > > > > > The
>>> >>
>>> >> > > > > > > >>>>> > attacker can render the resource completely
>>> >> unavailable.)"
>>> >>
>>> >> > > > > > > >>>>> > >
>>> >>
>>> >> > > > > > > >>>>> > > This vulnerability was patched in Java 6 Update
>>> 51,
>>> >>
>>> >> > > > > > > >>>>> > > but
>>> >>
>>> >> > > > post
>>> >>
>>> >> > > > > > end
>>> >>
>>> >> > > > > > > of
>>> >>
>>> >> > > > > > > >>>>> > life. Apple pushed out the update specifically
>>> because
>>> >>
>>> >> > > > > > > >>>>> > of
>>> >>
>>> >> > > > this
>>> >>
>>> >> > > > > > > >>>>> > vulnerability (http://support.apple.com/kb/HT5717)
>>> as
>>> >>
>>> >> > > > > > > >>>>> > did
>>> >>
>>> >> > > > some
>>> >>
>>> >> > > > > > > other
>>> >>
>>> >> > > > > > > >>>>> > vendors privately, but for the majority of people
>>> >>
>>> >> > > > > > > >>>>> > using
>>> >>
>>> >> > > Java
>>> >>
>>> >> > > > 6
>>> >>
>>> >> > > > > > > means they
>>> >>
>>> >> > > > > > > >>>>> > have a ticking time bomb.
>>> >>
>>> >> > > > > > > >>>>> > >
>>> >>
>>> >> > > > > > > >>>>> > > Allowing it to stay should be considered in terms
>>> of
>>> >>
>>> >> > > > > accepting
>>> >>
>>> >> > > > > > > the
>>> >>
>>> >> > > > > > > >>>>> whole
>>> >>
>>> >> > > > > > > >>>>> > risk posture.
>>> >>
>>> >> > > > > > > >>>>> > >
>>> >>
>>> >> > > > > > > >>>>> >
>>> >>
>>> >> > > > > > > >>>>> > There are some who get extended support, but I
>>> suspect
>>> >>
>>> >> > > > > > > >>>>> > many
>>> >>
>>> >> > > > > just
>>> >>
>>> >> > > > > > > have
>>> >>
>>> >> > > > > > > >>>>> > a if-it's-not-broke mentality when it comes to
>>> >>
>>> >> > > > > > > >>>>> > production
>>> >>
>>> >> > > > > > > deployments.
>>> >>
>>> >> > > > > > > >>>>> > The current code supports both java6 and java7 and
>>> so
>>> >>
>>> >> > > allows
>>> >>
>>> >> > > > > > these
>>> >>
>>> >> > > > > > > >>>>> > people to remain compatible, while enabling others
>>> to
>>> >>
>>> >> > > upgrade
>>> >>
>>> >> > > > > to
>>> >>
>>> >> > > > > > > the
>>> >>
>>> >> > > > > > > >>>>> > java7 runtime. This seems like the right compromise
>>> >>
>>> >> > > > > > > >>>>> > for a
>>> >>
>>> >> > > > > stable
>>> >>
>>> >> > > > > > > >>>>> > release series. Again, absolutely makes sense for
>>> >>
>>> >> > > > > > > >>>>> > trunk (ie
>>> >>
>>> >> > > > v3)
>>> >>
>>> >> > > > > > to
>>> >>
>>> >> > > > > > > >>>>> > require java7 or greater.
>>> >>
>>> >> > > > > > > >>>>> >
>>> >>
>>> >> > > > > > > >>>>>
>>> >>
>>> >> > > > > > > >>
>>> >>
>>> >> > > > > > > >>
>>> >>
>>> >> > > > > > > >>
>>> >>
>>> >> > > > > > > >> --
>>> >>
>>> >> > > > > > > >> - Tsuyoshi
>>> >>
>>> >> > > > > > >
>>> >>
>>> >> > > > > >
>>> >>
>>> >> > > > > >
>>> >>
>>> >> > > > > >
>>> >>
>>> >> > > > > > --
>>> >>
>>> >> > > > > > Best regards,
>>> >>
>>> >> > > > > >
>>> >>
>>> >> > > > > >    - Andy
>>> >>
>>> >> > > > > >
>>> >>
>>> >> > > > > > Problems worthy of attack prove their worth by hitting back. -
>>> >>
>>> >> > > > > > Piet
>>> >>
>>> >> > > > Hein
>>> >>
>>> >> > > > > > (via Tom White)
>>> >>
>>> >> > > > > >
>>> >>
>>> >> > > > >
>>> >>
>>> >> > > > > --
>>> >>
>>> >> > > > > CONFIDENTIALITY NOTICE
>>> >>
>>> >> > > > > NOTICE: This message is intended for the use of the individual
>>> or
>>> >>
>>> >> > > entity
>>> >>
>>> >> > > > to
>>> >>
>>> >> > > > > which it is addressed and may contain information that is
>>> >>
>>> >> > > > > confidential, privileged and exempt from disclosure under
>>> >>
>>> >> > > > > applicable law. If the
>>> >>
>>> >> > > reader
>>> >>
>>> >> > > > > of this message is not the intended recipient, you are hereby
>>> >>
>>> >> > > > > notified
>>> >>
>>> >> > > > that
>>> >>
>>> >> > > > > any printing, copying, dissemination, distribution, disclosure
>>> or
>>> >>
>>> >> > > > > forwarding of this communication is strictly prohibited. If you
>>> >>
>>> >> > > > > have received this communication in error, please contact the
>>> >>
>>> >> > > > > sender
>>> >>
>>> >> > > > immediately
>>> >>
>>> >> > > > > and delete it from your system. Thank You.
>>> >>
>>> >> > > > >
>>> >>
>>> >> > > >
>>> >>
>>> >> > >
>>> >>
>>> >> > > --
>>> >>
>>> >> > > CONFIDENTIALITY NOTICE
>>> >>
>>> >> > > NOTICE: This message is intended for the use of the individual or
>>> >>
>>> >> > > entity to which it is addressed and may contain information that is
>>> >>
>>> >> > > confidential, privileged and exempt from disclosure under applicable
>>> >>
>>> >> > > law. If the reader of this message is not the intended recipient,
>>> you
>>> >>
>>> >> > > are hereby notified that any printing, copying, dissemination,
>>> >>
>>> >> > > distribution, disclosure or forwarding of this communication is
>>> >>
>>> >> > > strictly prohibited. If you have received this communication in
>>> error,
>>> >>
>>> >> > > please contact the sender immediately and delete it from your
>>> system.
>>> >>
>>> >> > Thank You.
>>> >>
>>> >> > >
>>> >>
>>>
>>
>>
>>
>> --
>> Best regards,
>>
>>    - Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)

Mime
View raw message