hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raymie Stata <rst...@altiscale.com>
Subject Re: Plans of moving towards JDK7 in trunk
Date Tue, 08 Apr 2014 16:58:55 GMT
Is there broad consensus that, by end of 3Q2014 at the latest, "the
average" contributor to Hadoop should be free to use Java7 features?
And start pulling in libraries that have a Java7 dependency?  And
start doing the "janitorial" work of taking advantage of the Java7
APIs?  Or do we think that the bulk of Hadoop work will be done
against Java6 APIs (and avoiding Java7-dependent libraries) through
the end of the year?

If the consensus is that we introduce Java7 into the bulk of Hadoop
coding, what's the plan for getting there?  The answer can't be "right
now, in trunk."  Even if we agreed to start allowing Java7
dependencies into trunk, as a practical matter this isn't enough.
Right now, if I'm a random Hadoop contributor, I'd be stupid to
contribute to trunk: I know that any stable release in the near term
will be from branch2, so if I want a prayer of seeing my change in a
stable release, I'd better contribute to branch2.

If we want a path to allowing Java7 dependencies by Q4, then we need
one of the following:

1) "branch3 plan:" The major Hadoop vendors (you know who you are)
commit to shipping a "v3" of Hadoop in Q4 that allows Java7
dependencies and show signs of living up to that commitment (e.g., a
branch3 is created sometime soon).  This puts us all on a path towards
a "real" release of Hadoop that allows Java7 dependencies.

2) "branch2 plan:" deprecate Java6 as a runtime environment now,
publicly declare a time frame (e.g., 4Q2014) when _future development_
stops supporting Java6 runtime, and work with our customers in the
meantime to get them off a crazy-old version of Java (that's what
we're doing right now).

I don't see another path to allowing Java7 dependencies.  In the
current state of indecision, the smart programmer would be assuming no
Java7 dependencies into 2015.

On the one hand, I don't see the branch3 plan actually happening.
This is a big decision involving marketing, engineering, customer
support.  Plus it creates a problem for sales: Come summertime,
they'll have a hard time selling 2.x-based releases because they've
pre-announced support for 3.x.  It's just not going to happen.

On the other hand, I don't see the problem with the branch2 plan.  The
branch2 plan also requires the commitment from the major vendors, but
this decision is not nearly as galactic.  By the time 3Q2014 comes
along, this problem will be very rarified.  Also, don't forget that it
typically takes a customer 3-6 months to upgrade their Hadoop -- and a
customer who's afraid to shift off Java6 in 3Q2014 will probably take
a year to upgrade.  The branch2 plan implies a last Java6 release of
Hadoop in 3Q2014.  If we assume a Java7-averse customer will take a
year to upgrade to this release -- and then will take another year to
upgrade their cluster after that -- then they can be happily using
Java6 all the way into 2016.  (Another point, if 3Q2014 comes along
and vendors find they have so many customers still on Java6 that they
can't afford the discontinuity, then they can shift their MAJOR
version number of their product to communicate the discontinuity --
there's nothing that says that a vendor's versioning scheme must agree
exactly with Hadoop's.)

In short, we don't currently have a realistic path for introducing
Java7 dependencies into Hadoop.  Simply allowing them into trunk will
NOT solve this problem: any contributor who wants to see their code in
a stable release knows it'll have to flow through branch2 -- and thus
they'll have to avoid Java6 dependencies.  The branch2 plan is the
only plan proposed so far that gets us to Java7 dependencies by Q4.
And the important part of the branch2 plan is we make the decision
soon -- so we have time to notify folks and otherwise work that
decision out into the field.

  Raymie



On Tue, Apr 8, 2014 at 9:19 AM, Eli Collins <eli@cloudera.com> wrote:
> On Tue, Apr 8, 2014 at 2:00 AM, Ottenheimer, Davi
> <Davi.Ottenheimer@emc.com> wrote:
>>> From: Eli Collins [mailto:eli@cloudera.com]
>>> Sent: Monday, April 07, 2014 11:54 AM
>>>
>>>
>>> IMO we should not drop support for Java 6 in a minor update of a stable
>>> release (v2).  I don't think the larger Hadoop user base would find it
>>> acceptable that upgrading to a minor update caused their systems to stop
>>> working because they didn't upgrade Java. There are people still getting
>>> support for Java 6. ...
>>>
>>> Thanks,
>>> Eli
>>
>> Hi Eli,
>>
>> Technically you are correct those with extended support get critical security fixes
for 6 until the end of 2016. I am curious whether many of those are in the Hadoop user base.
Do you know? My guess is the vast majority are within Oracle's official public end of life,
which was over 12 months ago. Even Premier support ended Dec 2013:
>>
>> http://www.oracle.com/technetwork/java/eol-135779.html
>>
>> The end of Java 6 support carries much risk. It has to be considered in terms of
serious security vulnerabilities such as CVE-2013-2465 with CVSS score 10.0.
>>
>> http://www.cvedetails.com/cve/CVE-2013-2465/
>>
>> Since you mentioned "caused systems to stop" as an example of what would be a concern
to Hadoop users, please note the CVE-2013-2465 availability impact:
>>
>> "Complete (There is a total shutdown of the affected resource. The attacker can render
the resource completely unavailable.)"
>>
>> This vulnerability was patched in Java 6 Update 51, but post end of life. Apple pushed
out the update specifically because of this vulnerability (http://support.apple.com/kb/HT5717)
as did some other vendors privately, but for the majority of people using Java 6 means they
have a ticking time bomb.
>>
>> Allowing it to stay should be considered in terms of accepting the whole risk posture.
>>
>
> There are some who get extended support, but I suspect many just have
> a if-it's-not-broke mentality when it comes to production deployments.
> The current code supports both java6 and java7 and so allows these
> people to remain compatible, while enabling others to upgrade to the
> java7 runtime. This seems like the right compromise for a stable
> release series. Again, absolutely makes sense for trunk (ie v3) to
> require java7 or greater.

Mime
View raw message