hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: Plans of moving towards JDK7 in trunk
Date Mon, 14 Apr 2014 16:46:12 GMT
How well is trunk tested? Does anyone deploy it with real applications
running on top? When will the trunk codebase next be the basis for a
production release? An impromptu diff of hadoop-common trunk against
branch-2 as of today is 38,625 lines. Can they be said to be the same
animal? I ask because any disincentive toward putting code in trunk is
beside the point, if the only target worth pursuing today is branch-2
unless one doesn't care if the code is released for production use.
Questions on whither JDK6 or JDK7+ (or JRE6 versus JRE7+) only matter for
the vast majority of Hadoopers if talking about branch-2.


On Mon, Apr 14, 2014 at 9:22 AM, Colin McCabe <cmccabe@alumni.cmu.edu>wrote:

> I think the bottom line here is that as long as our stable release
> uses JDK6, there is going to be a very, very strong disincentive to
> put any code which can't run on JDK6 into trunk.
>
> Like I said earlier, the traditional reason for putting something in
> trunk but not the stable release is that it needs more testing.  If a
> stable release that drops support for JDK6 is more than a year away,
> does it make sense to put anything in trunk like that?  What might
> need more than a year of testing?  Certainly not changes to
> LocalFileSystem to use the new APIs.  I also don't think an upgrade to
> various libraries qualifies.
>
> It might be best to shelve this for now, like we've done in the past,
> until we're ready to talk about a stable release that requires JDK7+.
> At least that's my feeling.
>
> If we're really desperate for the new file APIs JDK7 provides, we
> could consider using loadable modules for it in branch-2.  This is
> similar to how we provide JNI versions of certain things on certain
> platforms, without dropping support for the other platforms.
>
> best,
> Colin
>
> On Sun, Apr 13, 2014 at 10:39 AM, Raymie Stata <rstata@altiscale.com>
> wrote:
> > There's an outstanding question addressed to me: "Are there particular
> > features or new dependencies that you would like to contribute (or see
> > contributed) that require using the Java 1.7 APIs?"  The question
> > misses the point: We'd figure out how to write something we wanted to
> > contribute to Hadoop against the APIs of Java4 if that's what it took
> > to get them into a stable release.  And at current course and speed,
> > that's how ridiculous things could get.
> >
> > To summarize, it seems like there's a vague consensus that it might be
> > okay to eventually allow the use of Java7 in trunk, but there's no
> > decision.  And there's been no answer to the concern that even if such
> > dependencies were allowed in Java7, the only people using them would
> > be people who uninterested in getting their patches into a stable
> > release of Hadoop on any knowable timeframe, which doesn't bode well
> > for the ability to stabilize that Java7 code when it comes time to
> > attempt to.
> >
> > I don't have more to add, so I'll go back to lurking.  It'll be
> > interesting to see where we'll be standing a year from now.
> >
> > On Sun, Apr 13, 2014 at 2:09 AM, Tsuyoshi OZAWA
> > <ozawa.tsuyoshi@gmail.com> wrote:
> >> Hi,
> >>
> >> +1 for Karthik's idea(non-binding).
> >>
> >> IMO, we should keep the compatibility between JDK 6 and JDK 7 on both
> branch-1
> >> and branch-2, because users can be using them. For future releases that
> we can
> >> declare breaking compatibility(e.g. 3.0.0 release), we can use JDK 7
> >> features if we
> >> can get benefits. However, it can increase maintenance costs and
> distributes the
> >> efforts of contributions to maintain branches. Then, I think it is
> >> reasonable approach
> >> that we use limited and minimum JDK-7 APIs when we have reasons we need
> to use
> >> the features.
> >> By the way, if we start to use JDK 7 APIs, we should declare the basis
> >> when to use
> >> JDK 7 APIs on Wiki not to confuse contributors.
> >>
> >> Thanks,
> >> - Tsuyoshi
> >>
> >> On Wed, Apr 9, 2014 at 11:44 AM, Raymie Stata <rstata@altiscale.com>
> wrote:
> >>>> It might make sense to try to enumerate the benefits of switching to
> >>>> Java7 APIs and dependencies.
> >>>
> >>>   - Java7 introduced a huge number of language, byte-code, API, and
> >>> tooling enhancements!  Just to name a few: try-with-resources, newer
> >>> and stronger encyrption methods, more scalable concurrency primitives.
> >>>  See http://www.slideshare.net/boulderjug/55-things-in-java-7
> >>>
> >>>   - We can't update current dependencies, and we can't add cool new
> ones.
> >>>
> >>>   - Putting language/APIs aside, don't forget that a huge amount of
> effort
> >>> goes into qualifying for Java6 (at least, I hope the folks claiming to
> >>> support Java6 are putting in such an effort :-).  Wouldn't Hadoop
> >>> users/customers be better served if qualification effort went into
> >>> Java7/8 versus Java6/7?
> >>>
> >>> Getting to Java7 as a development env (and Java8 as a runtime env)
> >>> seems like a no-brainer.  Question is: How?
> >>>
> >>> On Tue, Apr 8, 2014 at 10:21 AM, Sandy Ryza <sandy.ryza@cloudera.com>
> wrote:
> >>>> It might make sense to try to enumerate the benefits of switching to
> Java7
> >>>> APIs and dependencies.  IMO, the ones listed so far on this thread
> don't
> >>>> make a compelling enough case to drop Java6 in branch-2 on any time
> frame,
> >>>> even if this means supporting Java6 through 2015.  For example, the
> change
> >>>> in RawLocalFileSystem semantics might be an incompatible change for
> >>>> branch-2 any way.
> >>>>
> >>>>
> >>>> On Tue, Apr 8, 2014 at 10:05 AM, Karthik Kambatla <kasha@cloudera.com
> >wrote:
> >>>>
> >>>>> +1 to NOT breaking compatibility in branch-2.
> >>>>>
> >>>>> I think it is reasonable to require JDK7 for trunk, if we limit
use
> of
> >>>>> JDK7-only API to security fixes etc. If we make other optimizations
> (like
> >>>>> IO), it would be a pain to backport things to branch-2. I guess
this
> all
> >>>>> depends on when we see ourselves shipping Hadoop-3. Any ideas on
> that?
> >>>>>
> >>>>>
> >>>>> On Tue, Apr 8, 2014 at 9:19 AM, Eli Collins <eli@cloudera.com>
> wrote:
> >>>>>
> >>>>> > On Tue, Apr 8, 2014 at 2:00 AM, Ottenheimer, Davi
> >>>>> > <Davi.Ottenheimer@emc.com> wrote:
> >>>>> > >> From: Eli Collins [mailto:eli@cloudera.com]
> >>>>> > >> Sent: Monday, April 07, 2014 11:54 AM
> >>>>> > >>
> >>>>> > >>
> >>>>> > >> IMO we should not drop support for Java 6 in a minor
update of a
> >>>>> stable
> >>>>> > >> release (v2).  I don't think the larger Hadoop user
base would
> find it
> >>>>> > >> acceptable that upgrading to a minor update caused
their
> systems to
> >>>>> stop
> >>>>> > >> working because they didn't upgrade Java. There are
people still
> >>>>> getting
> >>>>> > >> support for Java 6. ...
> >>>>> > >>
> >>>>> > >> Thanks,
> >>>>> > >> Eli
> >>>>> > >
> >>>>> > > Hi Eli,
> >>>>> > >
> >>>>> > > Technically you are correct those with extended support
get
> critical
> >>>>> > security fixes for 6 until the end of 2016. I am curious whether
> many of
> >>>>> > those are in the Hadoop user base. Do you know? My guess is
the
> vast
> >>>>> > majority are within Oracle's official public end of life, which
> was over
> >>>>> 12
> >>>>> > months ago. Even Premier support ended Dec 2013:
> >>>>> > >
> >>>>> > > http://www.oracle.com/technetwork/java/eol-135779.html
> >>>>> > >
> >>>>> > > The end of Java 6 support carries much risk. It has to
be
> considered in
> >>>>> > terms of serious security vulnerabilities such as CVE-2013-2465
> with CVSS
> >>>>> > score 10.0.
> >>>>> > >
> >>>>> > > http://www.cvedetails.com/cve/CVE-2013-2465/
> >>>>> > >
> >>>>> > > Since you mentioned "caused systems to stop" as an example
of
> what
> >>>>> would
> >>>>> > be a concern to Hadoop users, please note the CVE-2013-2465
> availability
> >>>>> > impact:
> >>>>> > >
> >>>>> > > "Complete (There is a total shutdown of the affected resource.
> The
> >>>>> > attacker can render the resource completely unavailable.)"
> >>>>> > >
> >>>>> > > This vulnerability was patched in Java 6 Update 51, but
post end
> of
> >>>>> > life. Apple pushed out the update specifically because of this
> >>>>> > vulnerability (http://support.apple.com/kb/HT5717) as did some
> other
> >>>>> > vendors privately, but for the majority of people using Java
6
> means they
> >>>>> > have a ticking time bomb.
> >>>>> > >
> >>>>> > > Allowing it to stay should be considered in terms of accepting
> the
> >>>>> whole
> >>>>> > risk posture.
> >>>>> > >
> >>>>> >
> >>>>> > There are some who get extended support, but I suspect many
just
> have
> >>>>> > a if-it's-not-broke mentality when it comes to production
> deployments.
> >>>>> > The current code supports both java6 and java7 and so allows
these
> >>>>> > people to remain compatible, while enabling others to upgrade
to
> the
> >>>>> > java7 runtime. This seems like the right compromise for a stable
> >>>>> > release series. Again, absolutely makes sense for trunk (ie
v3) to
> >>>>> > require java7 or greater.
> >>>>> >
> >>>>>
> >>
> >>
> >>
> >> --
> >> - Tsuyoshi
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message