hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinayakumar B <vinayakum...@apache.org>
Subject Re: Plans of moving towards JDK7 in trunk
Date Thu, 10 Apr 2014 04:14:48 GMT
+1 for keeping jdk 6 suppprt in branch-2 and start using jdk 7 in trunk.

I agree that this approach makes patch generation difficult for branch-2
and trunk.

Also the actual benefit and real issues after start using jdk7 will be
known only if atleast one of the release is out in trunk version.

Regards,
Vinay
I think this thread isn't so much about whether java7, 8 etc features
are valuable, they are useful of course, and we'll want to adopt them,
it's a question of how we adopt them and in which releases.

For the sake of this discussion we should separate the runtime from
the programming APIs. Users are already migrating to the java7 runtime
for most of the reasons listed below (support, performance, bugs,
etc), and the various distributions cert their Hadoop 2 based
distributions on java7.  This gives users many of the benefits of
java7, without forcing users off java6. Ie Hadoop does not need to
switch to the java7 programming APIs to make sure everyone has a
supported runtime.

The question here is really about when Hadoop, and the Hadoop
ecosystem (since adjacent projects often end up in the same classpath)
start using the java7 programming APIs and therefore break
compatibility with java6 runtimes. I think our java6 runtime users
would consider dropping support for their java runtime in an update of
a major release to be an incompatible change (the binaries stop
working on their current jvm). That may be worth it if we can
articulate sufficient value to offset the cost (they have to upgrade
their environment, might make rolling upgrades stop working, etc), but
I've not yet heard an argument that articulates the value relative to
the cost.  Eg upgrading to the java7 APIs allows us to pull in
dependencies with new major versions, but only if those dependencies
don't break compatibility (which is likely given that our classpaths
aren't so isolated), and, realistically, only if the entire Hadoop
stack moves to java7 as well (eg we have to recompile HBase to
generate v1.7 binaries even if they stick on API v1.6). I'm not aware
of a feature, bug etc that really motivates this.

An alternate approach is to keep the current stable release series
(v2.x) as is, and start using new APIs in trunk (for v3). This will be
a major upgrade for Hadoop and therefore an incompatible change like
this is to be expected (it would be great if this came with additional
changes to better isolate classpaths and dependencies from each
other). It allows us to continue to support multiple types of users
with different branches, vs forcing all users onto a new version. It
of course means that 2.x users will not get the benefits of the new
API, but its unclear what those benefits are given they can already
get the benefits of adopting the newer java runtimes today.

Thanks,
Eli


On Wed, Apr 9, 2014 at 9:38 AM, Andrew Purtell <apurtell@apache.org> wrote:
> A Java 8 runtime would also offer transparent performance improvements
like
> a reimplementation of ConcurrentSkipListMap, C2 support for AES cipher
> acceleration with native CPU instructions, perf improvements for going
from
> String to byte[] or vice versa, and IIRC after 8u20 monitor lock elision
> using restricted transactional memory with hardware support (if
available).
> Getting away from fully transparent changes but tractable to deal with
> using reflection, removal of the permanent generation, support for AEAD
> cipher modes like AES-GCM, stronger cipher and key exchange algorithms,
TLS
> 1.2, support for some krb 5 features not handled previously.
>
>
>
> On Tue, Apr 8, 2014 at 7:44 PM, Raymie Stata <rstata@altiscale.com> wrote:
>
>> > It might make sense to try to enumerate the benefits of switching to
>> > Java7 APIs and dependencies.
>>
>>   - Java7 introduced a huge number of language, byte-code, API, and
>> tooling enhancements!  Just to name a few: try-with-resources, newer
>> and stronger encyrption methods, more scalable concurrency primitives.
>>  See http://www.slideshare.net/boulderjug/55-things-in-java-7
>>
>>   - We can't update current dependencies, and we can't add cool new ones.
>>
>>   - Putting language/APIs aside, don't forget that a huge amount of
effort
>> goes into qualifying for Java6 (at least, I hope the folks claiming to
>> support Java6 are putting in such an effort :-).  Wouldn't Hadoop
>> users/customers be better served if qualification effort went into
>> Java7/8 versus Java6/7?
>>
>> Getting to Java7 as a development env (and Java8 as a runtime env)
>> seems like a no-brainer.  Question is: How?
>>
>> On Tue, Apr 8, 2014 at 10:21 AM, Sandy Ryza <sandy.ryza@cloudera.com>
>> wrote:
>> > It might make sense to try to enumerate the benefits of switching to
>> Java7
>> > APIs and dependencies.  IMO, the ones listed so far on this thread
don't
>> > make a compelling enough case to drop Java6 in branch-2 on any time
>> frame,
>> > even if this means supporting Java6 through 2015.  For example, the
>> change
>> > in RawLocalFileSystem semantics might be an incompatible change for
>> > branch-2 any way.
>> >
>> >
>> > On Tue, Apr 8, 2014 at 10:05 AM, Karthik Kambatla <kasha@cloudera.com
>> >wrote:
>> >
>> >> +1 to NOT breaking compatibility in branch-2.
>> >>
>> >> I think it is reasonable to require JDK7 for trunk, if we limit use of
>> >> JDK7-only API to security fixes etc. If we make other optimizations
>> (like
>> >> IO), it would be a pain to backport things to branch-2. I guess this
all
>> >> depends on when we see ourselves shipping Hadoop-3. Any ideas on that?
>> >>
>> >>
>> >> On Tue, Apr 8, 2014 at 9:19 AM, Eli Collins <eli@cloudera.com> wrote:
>> >>
>> >> > On Tue, Apr 8, 2014 at 2:00 AM, Ottenheimer, Davi
>> >> > <Davi.Ottenheimer@emc.com> wrote:
>> >> > >> From: Eli Collins [mailto:eli@cloudera.com]
>> >> > >> Sent: Monday, April 07, 2014 11:54 AM
>> >> > >>
>> >> > >>
>> >> > >> IMO we should not drop support for Java 6 in a minor update
of a
>> >> stable
>> >> > >> release (v2).  I don't think the larger Hadoop user base would
>> find it
>> >> > >> acceptable that upgrading to a minor update caused their systems
to
>> >> stop
>> >> > >> working because they didn't upgrade Java. There are people
still
>> >> getting
>> >> > >> support for Java 6. ...
>> >> > >>
>> >> > >> Thanks,
>> >> > >> Eli
>> >> > >
>> >> > > Hi Eli,
>> >> > >
>> >> > > Technically you are correct those with extended support get
critical
>> >> > security fixes for 6 until the end of 2016. I am curious whether
many
>> of
>> >> > those are in the Hadoop user base. Do you know? My guess is the vast
>> >> > majority are within Oracle's official public end of life, which was
>> over
>> >> 12
>> >> > months ago. Even Premier support ended Dec 2013:
>> >> > >
>> >> > > http://www.oracle.com/technetwork/java/eol-135779.html
>> >> > >
>> >> > > The end of Java 6 support carries much risk. It has to be
>> considered in
>> >> > terms of serious security vulnerabilities such as CVE-2013-2465 with
>> CVSS
>> >> > score 10.0.
>> >> > >
>> >> > > http://www.cvedetails.com/cve/CVE-2013-2465/
>> >> > >
>> >> > > Since you mentioned "caused systems to stop" as an example of
what
>> >> would
>> >> > be a concern to Hadoop users, please note the CVE-2013-2465
>> availability
>> >> > impact:
>> >> > >
>> >> > > "Complete (There is a total shutdown of the affected resource.
The
>> >> > attacker can render the resource completely unavailable.)"
>> >> > >
>> >> > > This vulnerability was patched in Java 6 Update 51, but post end
of
>> >> > life. Apple pushed out the update specifically because of this
>> >> > vulnerability (http://support.apple.com/kb/HT5717) as did some other
>> >> > vendors privately, but for the majority of people using Java 6 means
>> they
>> >> > have a ticking time bomb.
>> >> > >
>> >> > > Allowing it to stay should be considered in terms of accepting
the
>> >> whole
>> >> > risk posture.
>> >> > >
>> >> >
>> >> > There are some who get extended support, but I suspect many just
have
>> >> > a if-it's-not-broke mentality when it comes to production
deployments.
>> >> > The current code supports both java6 and java7 and so allows these
>> >> > people to remain compatible, while enabling others to upgrade to the
>> >> > java7 runtime. This seems like the right compromise for a stable
>> >> > release series. Again, absolutely makes sense for trunk (ie v3) to
>> >> > require java7 or greater.
>> >> >
>> >>
>>
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message