hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prasanth Jayachandran <pjayachand...@hortonworks.com>
Subject Re: [discuss] jdk8 support
Date Fri, 03 Jun 2016 17:39:50 GMT
+1 for using jdk8 as minimum required version.

Thanks
Prasanth




On Fri, Jun 3, 2016 at 10:21 AM -0700, "Siddharth Seth" <sseth@apache.org<mailto:sseth@apache.org>>
wrote:

+1. Drop Java7 support, and start using JDK8 features.

On Fri, Jun 3, 2016 at 8:13 AM, Ashutosh Chauhan <hashutosh@apache.org>
wrote:

> What I meant was that we start compiling using jdk8 as well. That will
> allow devs to use jdk8 only features (lambda functions etc.)
>
> On Fri, Jun 3, 2016 at 8:10 AM, Sergio Pena <sergio.pena@cloudera.com>
> wrote:
>
> > Hey Ashutosh,
> >
> > I switched to JDK8 in master last weekend. Jenkins has been running all
> > tests in Java8 successfully for a week now.
> > There are still a few tests we need to fix, but so far is looking good.
> >
> > Sergio
> >
> > On Fri, Jun 3, 2016 at 12:02 AM, Ashutosh Chauhan <hashutosh@apache.org>
> > wrote:
> >
> > > Now that branch-2.1 has been cut, I think its an opportune time to drop
> > > support of jdk7 altogether in master. That is we compile using jdk8 as
> > > well. What do others think?
> > >
> > > Ashutosh
> > >
> > > On Fri, May 27, 2016 at 7:41 PM, Sergio Pena <sergio.pena@cloudera.com
> >
> > > wrote:
> > >
> > > > I did the change to JDK8 on Ptest now.
> > > >
> > > > Please let me tknow if there are issues with Java8.
> > > >
> > > > - Sergio
> > > >
> > > > On Fri, May 27, 2016 at 2:45 PM, Sergio Pena <
> sergio.pena@cloudera.com
> > >
> > > > wrote:
> > > >
> > > > > Thanks Mohit.
> > > > >
> > > > > I will plan to do the JDK8 change on Jenkins today EOD and monitor
> > any
> > > > > issues through the weekend.
> > > > >
> > > > > - Sergio
> > > > >
> > > > >
> > > > > On Fri, May 27, 2016 at 2:58 AM, Mohit Sabharwal <
> mohit@cloudera.com
> > >
> > > > > wrote:
> > > > >
> > > > >> Update on moving Hive2 tests to JDK8: I've addressed almost all
> test
> > > > >> failures in HIVE-13547 on java8 branch. There is one remaining
> > > > >> open item (HIVE-13834) which is currently assigned. Given current
> > > > >> state of flaky test runs, there might be few more.
> > > > >>
> > > > >> I will work with Sergio to merge the test fixes to master and
> switch
> > > > >> the Hive2 pre-commit job to use JDK8, hopefully sometime tomorrow.
> > > > >>
> > > > >> After Hive2 tests switch, if your patch sees ordering related
test
> > > > >> failures in pre-commit run, it's likely JDK version related and
> > you'll
> > > > >> need to build & re-run the test using JDK8. Number of such
tests
> > > > >> should be relatively small.
> > > > >>
> > > > >> On Tue, Apr 19, 2016 at 10:43 AM, Mohit Sabharwal <
> > mohit@cloudera.com
> > > >
> > > > >> wrote:
> > > > >>
> > > > >> > Created HIVE-13547 to track switching 2x tests to JDK8.
> > > > >> >
> > > > >> > On Wed, Apr 13, 2016 at 10:02 AM, Sergio Pena <
> > > > sergio.pena@cloudera.com
> > > > >> >
> > > > >> > wrote:
> > > > >> >
> > > > >> >> I agree with such change as JDK7 is not longer supported.
> > > > >> >>
> > > > >> >> Changes on Jenkins and Hive PTest shouldn't be hard.
We just
> need
> > > to
> > > > >> >> replace the path from java7 to java8. But I think we
should fix
> > all
> > > > >> JDK8
> > > > >> >> issues or most of them before doing the change or we
will end
> up
> > > > >> having a
> > > > >> >> lot of failures on all JIRAs running pre-commit tests.
> > > > >> >>
> > > > >> >> +1 with the change.
> > > > >> >>
> > > > >> >> On Mon, Apr 11, 2016 at 9:34 PM, Siddharth Seth <
> > sseth@apache.org>
> > > > >> wrote:
> > > > >> >>
> > > > >> >> > Option 3 sounds good. I'd ideally like to make
JDK8 the
> minimum
> > > > >> >> requirement
> > > > >> >> > soon as well.
> > > > >> >> >
> > > > >> >> > On Mon, Apr 11, 2016 at 4:59 PM, Szehon Ho <
> > szehon@cloudera.com>
> > > > >> wrote:
> > > > >> >> >
> > > > >> >> > > Sounds like a good plan, +1
> > > > >> >> > >
> > > > >> >> > > On Mon, Apr 11, 2016 at 4:31 PM, Mohit Sabharwal
<
> > > > >> mohit@cloudera.com>
> > > > >> >> > > wrote:
> > > > >> >> > >
> > > > >> >> > > > Thanks, Ashutosh. Makes sense to keep
the source and
> target
> > > as
> > > > >> 1.7
> > > > >> >> > since
> > > > >> >> > > > we're not using any JDK8 specific features
yet. So,
> option
> > > (3)
> > > > >> >> > > essentially
> > > > >> >> > > > just means using JDK8 exclusively to
build & test Hive2.
> > > > >> >> > > >
> > > > >> >> > > > On Sat, Apr 9, 2016 at 12:23 PM, Ashutosh
Chauhan <
> > > > >> >> > hashutosh@apache.org>
> > > > >> >> > > > wrote:
> > > > >> >> > > >
> > > > >> >> > > > > Hi Mohit,
> > > > >> >> > > > >
> > > > >> >> > > > > I also think option 3 makes sense.
We should strive to
> > keep
> > > > >> test
> > > > >> >> > matrix
> > > > >> >> > > > > small so that we can do fast QA
runs for dev patches.
> > > > >> >> > > > > We can just use jdk7 to build &
test Hive1 and jdk8 to
> > > build
> > > > &
> > > > >> >> test
> > > > >> >> > > > Hive2.
> > > > >> >> > > > > However, I am not sure of explicitly
dropping support
> > > > >> altogether
> > > > >> >> for
> > > > >> >> > > jdk7
> > > > >> >> > > > > on Hive2. We should make sure that
in pom.xml java
> > source &
> > > > >> target
> > > > >> >> > > > > compatibility is still 1.7 (which
already is the case
> > > > >> currently)
> > > > >> >> so
> > > > >> >> > > that
> > > > >> >> > > > > Hive2 is still compatible with jdk7.
Unit tests as I
> said
> > > we
> > > > >> can
> > > > >> >> run
> > > > >> >> > on
> > > > >> >> > > > > jdk8.
> > > > >> >> > > > >
> > > > >> >> > > > > Thanks,
> > > > >> >> > > > > Ashutosh
> > > > >> >> > > > >
> > > > >> >> > > > > On Fri, Apr 8, 2016 at 4:34 PM,
Mohit Sabharwal <
> > > > >> >> mohit@cloudera.com>
> > > > >> >> > > > > wrote:
> > > > >> >> > > > >
> > > > >> >> > > > > > Hi folks,
> > > > >> >> > > > > >
> > > > >> >> > > > > > Oracle EOL'ed (ended public
updates) for Java 7 in
> > April
> > > > >> 2015.
> > > > >> >> > > > > > In order to support Java 8,
we fixed quite a bunch of
> > > tests
> > > > >> in
> > > > >> >> > > > > > HIVE-8607 (*) early last year.
However, since our
> > > > pre-commit
> > > > >> >> tests
> > > > >> >> > > run
> > > > >> >> > > > > > on JDK7 only, any JDK8 test
failures are getting
> > ignored.
> > > > As
> > > > >> a
> > > > >> >> > > result,
> > > > >> >> > > > > > the count has crept up
> > > > >> >> > > > > > <
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/HIVE-TRUNK-JAVA8/
> > > > >> >> > > > > > >
> > > > >> >> > > > > > back
> > > > >> >> > > > > > from zero to 125.
> > > > >> >> > > > > >
> > > > >> >> > > > > > Some options to address this:
> > > > >> >> > > > > >
> > > > >> >> > > > > > (1) Run pre-commit tests on
both JDK7 and JDK8, in
> both
> > > 1.x
> > > > >> >> > > > > > and 2.x: This will further
slow down the pre-commit
> > run.
> > > > >> >> > > > > >
> > > > >> >> > > > > > (2) Alternate pre-commit test
runs between JDK7 and
> > JDK8
> > > > (in
> > > > >> >> > > > > > both 1.x and 2.x): It's a cheap
hack. More messy,
> > though
> > > > >> >> failures
> > > > >> >> > > > > > won't get ignored.
> > > > >> >> > > > > >
> > > > >> >> > > > > > (3) Drop support for JDK7 in
Hive 2.x, i.e. build and
> > > test
> > > > >> Hive
> > > > >> >> 2.x
> > > > >> >> > > on
> > > > >> >> > > > > > JDK8 only. For Hive 1.x, continue
running JDK7
> > pre-commit
> > > > >> tests.
> > > > >> >> > > > > > The pros are:
> > > > >> >> > > > > > - Simple test matrix.
> > > > >> >> > > > > > - No need to slow down test
run or maintain version
> > > > specific
> > > > >> >> golden
> > > > >> >> > > > files
> > > > >> >> > > > > > (**).
> > > > >> >> > > > > > - 2.x looks like the logical
place to move to JDK8.
> > > > >> >> > > > > > - Users transitioning to JDK8
for all other services
> do
> > > not
> > > > >> >> have to
> > > > >> >> > > > > > maintain
> > > > >> >> > > > > > multiple java versions on the
cluster.
> > > > >> >> > > > > >
> > > > >> >> > > > > > Option (3) looks most attractive
to me.
> > > > >> >> > > > > >
> > > > >> >> > > > > > Moving to JDK8 also lines us
up better for Java 9,
> > which
> > > is
> > > > >> on
> > > > >> >> the
> > > > >> >> > > > > > horizon (Oracle will end public
updates for Java 8 in
> > Sep
> > > > >> 2017)
> > > > >> >> > > > > >
> > > > >> >> > > > > > Around 100 of the latest crop
of failures are due to
> > one
> > > > >> cause
> > > > >> >> > > > > > (HIVE-13409).
> > > > >> >> > > > > > I can take a pass at triaging
the rest if there is
> > > > consensus
> > > > >> >> around
> > > > >> >> > > > what
> > > > >> >> > > > > > to do overall.
> > > > >> >> > > > > >
> > > > >> >> > > > > > Thanks,
> > > > >> >> > > > > > Mohit
> > > > >> >> > > > > >
> > > > >> >> > > > > > (*) Most test failures are
due to JDK8 using a
> > different
> > > > hash
> > > > >> >> > > function
> > > > >> >> > > > > for
> > > > >> >> > > > > > HashMap  compared to JDK7.
This results in (mostly
> > > benign,
> > > > >> but
> > > > >> >> hard
> > > > >> >> > > > > > to debug) ordering differences
in q-file output
> related
> > > to
> > > > >> >> > > > serialization
> > > > >> >> > > > > > order of map entries, numbering
of stages in query
> > plan,
> > > > etc.
> > > > >> >> > > > > >
> > > > >> >> > > > > > (**) In some cases, hash function
related ordering
> > > > >> differences
> > > > >> >> > > > originate
> > > > >> >> > > > > > in external libraries like
Avro, antlr, json
> ,Thrift's
> > > map,
> > > > >> etc.
> > > > >> >> > for
> > > > >> >> > > > > which
> > > > >> >> > > > > > code
> > > > >> >> > > > > > changes are either more involved
or led to more test
> > > > >> failures.
> > > > >> >> To
> > > > >> >> > > > > address
> > > > >> >> > > > > > this,
> > > > >> >> > > > > > we added support for version
specific golden files
> > > > >> (HIVE-9109).
> > > > >> >> > > > > Currently,
> > > > >> >> > > > > > there
> > > > >> >> > > > > > are ~40 golden files with ".java1.8.out"
extension.
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >> >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message